-- Logs begin at Fri 2022-06-24 09:56:55 UTC, end at Fri 2022-06-24 10:57:00 UTC. -- Jun 24 10:21:32.194264 dv-cinder-847536-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 10:21:32.207573 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 10:21:32 2022] *** Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: nodename: dv-cinder-847536-1-iscsi Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: machine: x86_64 Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: clock source: unix Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: pcre jit disabled Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: detected number of CPU cores: 4 Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: current working directory: / Jun 24 10:21:32.208286 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: detected binary path: /usr/bin/uwsgi-core Jun 24 10:21:32.208851 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: your processes number limit is 47680 Jun 24 10:21:32.208851 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: your memory page size is 4096 bytes Jun 24 10:21:32.208851 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: detected max file descriptor number: 2048 Jun 24 10:21:32.208851 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: lock engine: pthread robust mutexes Jun 24 10:21:32.208851 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: thunder lock: enabled Jun 24 10:21:32.208851 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 10:21:32.208851 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 10:21:32.252150 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: Python main interpreter initialized at 0x5636c58ec9c0 Jun 24 10:21:32.252150 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: python threads support enabled Jun 24 10:21:32.252150 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: your server socket listen backlog is limited to 100 connections Jun 24 10:21:32.252150 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: your mercy for graceful operations on workers is 90 seconds Jun 24 10:21:32.252541 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 10:21:32.252541 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: *** Operational MODE: preforking *** Jun 24 10:21:32.252864 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: *** uWSGI is running in multiple interpreter mode *** Jun 24 10:21:32.252864 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: spawned uWSGI master process (pid: 80130) Jun 24 10:21:32.253036 dv-cinder-847536-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 10:21:32.253873 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: spawned uWSGI worker 1 (pid: 80145, cores: 1) Jun 24 10:21:32.253962 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: spawned uWSGI worker 2 (pid: 80146, cores: 1) Jun 24 10:21:32.255359 dv-cinder-847536-1-iscsi devstack@keystone.service[80130]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 10:21:37.627202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 10:21:37.627202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: from cryptography.utils import int_from_bytes Jun 24 10:21:37.627994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 10:21:37.627994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: from cryptography.utils import int_from_bytes Jun 24 10:21:37.628406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 10:21:37.628406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: from cryptography.utils import int_from_bytes Jun 24 10:21:37.628534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 10:21:37.628534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: from cryptography.utils import int_from_bytes Jun 24 10:21:37.675799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 10:21:37.675993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 10:21:37.676075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 10:21:37.676147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] command line args: [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 10:21:37.676206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 10:21:37.676345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] command line args: [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 10:21:37.676428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 10:21:37.676542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ================================================================================ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 10:21:37.676621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 10:21:37.676841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ================================================================================ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 10:21:37.676976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] admin_token = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.677058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] config_dir = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.677128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] admin_token = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.677256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.677339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config_dir = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.677403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] config_source = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.677458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.677590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] debug = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.677675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config_source = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.677787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] debug = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_config_append = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.679985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_config_append = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680046 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_dir = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_dir = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_file = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.680960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.681834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.682019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.682133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.682242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.682316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.682516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.682642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.682729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.682798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.683008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.683131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.683497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.683626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.683626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] publish_errors = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.683778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] publish_errors = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.683846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.683912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.684100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.684229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.684229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.684374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.684475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.684579 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.684829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.684960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] standard_threads = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] standard_threads = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685299 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_journal = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.685998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_journal = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.686070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_json = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.686136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_json = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.686376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_stderr = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.686451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_stderr = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.686529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_syslog = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.686605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_syslog = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.686679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.686748 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:21:37.686852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.686941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.687105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.687223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.687374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.687493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.687570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.687648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.687834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.687964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.688048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.688123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.688299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.688419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.688508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.688581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.688766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.688885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.689020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.689100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.689263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.689343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.689495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.689574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.689737 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.689815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.689968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.690053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.690208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.690286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.690450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.690540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.690685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.690756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.690921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.691000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.691169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.691246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.691408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.691492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.691641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.691719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.691887 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.691964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.692142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.692221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.692392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.692471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.692635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.692720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.692867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.692943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.693096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.693169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.693325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.693398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.693553 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.693628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.693797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.693870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.694038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.694120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.694264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.694337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.694491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.694570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.694734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.694818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.694977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.695050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.695209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.695282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.695453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.695540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.695679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.695752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.695912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.695987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.696143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.696296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.696368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.696434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.696586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.696662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.696838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.696927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.697070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.697150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.697304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.697377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.697548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.697633 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.697778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.697851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.698008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.698081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.698237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.698317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.698467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.698532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.698697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.698761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.698940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.699015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.699268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.699344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.699410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.699465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.699613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.699701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.699847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.699975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.700093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.700209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.700337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.700466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.700591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.700702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.700848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.700974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.701092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.701220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.701934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.702059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.702149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.702263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.702345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.702511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.702628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.702750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.702867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.702983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.703105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.703222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.703334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.703776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.703895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.704022 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.704140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.704257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.704379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.704505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.704629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.704763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.704883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.705006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.705137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.705258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.705378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.705504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.705621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.705750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.705878 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.706001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.706126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.706246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.706366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.706487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.706617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.706738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.706738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.706955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.707113 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.707238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.707383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.707505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.707630 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.707756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.707876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.707995 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.708122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.708240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.708359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.708481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.708600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.708720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.708988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.709111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.709242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.709362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.709486 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.709604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.709736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.709864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.709992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.710111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.710231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.710350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.710470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.710591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.710718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.710836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.710971 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.711091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.711210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.711335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.711462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.711584 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.711703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.711833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.711954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.712073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.712200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.712319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.712439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.712569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.712690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.712832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.712964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.external = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.713084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.external = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.713204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.713323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.713442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.713595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.713731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.713864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.713991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.password = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.714111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.password = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.714231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.token = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.714360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.token = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.714480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.714610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.714732 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.714852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.714972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.715092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.715212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.715343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.715463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.715585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.715706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.715825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.715954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.716080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.716200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.716319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.716439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.716511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.716631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.716806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.716942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.717070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.717192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.717310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.717432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.717562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.717694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.717815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.717937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.718057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.718178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.718305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.718436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.718558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.718685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.718806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.718928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.719055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.719182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.719318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.719440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.719559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.719679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.719807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.719929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.719929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.720170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.720291 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.720417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.720548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.720669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.720803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.720928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.721060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.721183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.721311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.721433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.721554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.721674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.721795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.721915 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.722044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.722165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.722286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.722407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.722528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.722658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.722789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.722908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.723029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.723151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.723271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.723392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.723583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.723704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.723824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.723945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.724066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.724195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.724316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.724447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.724569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.724689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.724831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.724960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.725078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.725197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.725314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.725431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.725549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.725673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.725796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.725917 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.726034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.726162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.726280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.726406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.726524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.726641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.726758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.726875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.726997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.727127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.727250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.727369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.727369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.727600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.727600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.727839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.727956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.728079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.728201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.728318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.728436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.728560 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.728681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.728855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.728974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.729094 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.729215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.729346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.729482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.729603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.729681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.729835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.729956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.730084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.730211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.730332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.730455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.730574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.730694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.730822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.730942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.731072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.731193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.731313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.731435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.731562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.731683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.731805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.731925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.732044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.732164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.732291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.732413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.732557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.732713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.732855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.732988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.733168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.733310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.733429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.733548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.733666 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.733783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.733909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.734027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.734144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.734271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.734389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.734507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.734642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.734761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.734878 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.734996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.735113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.735231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.735356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.735475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.735593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.735711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.735828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.735946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.736071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.736206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.736324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.736442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.736566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.736692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.736853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.736974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.737095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.737216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.737336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.737588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.737708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.737830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.737961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.738082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.738210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.738331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.738453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.738573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.738693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.738815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.738948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.739069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.739189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.739309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.739430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.739566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.739715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.739833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.739955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.740111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.740229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.740350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.740479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.740600 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.740720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.740854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.740973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.741091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.741222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.741350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.741470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.741587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.741704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.741821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.741947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.742066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.742184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.742302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.742420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.742538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.742663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.742781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.742899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.743026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.743144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.743261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.743387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.743506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.743624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.743741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.743859 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.743976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.744102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.744220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.744339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.744457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.744574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.744706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.744854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.744975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.745092 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.745209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.745327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.745444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.745569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.745687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.745805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.745922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.746039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.746156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.746291 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.746409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.746527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.746644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.746762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.746879 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.747071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.747191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.747310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.747427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.747545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.747663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] role.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.747788 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.747907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] role.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.748039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.748174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.748255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.748314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.748441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.748560 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.748679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.748852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.748975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.749096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.749227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.749353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.749476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.749607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.749728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.749856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.749976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.750096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.750216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.750337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.750456 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.750585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.750706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.750828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.750950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.751070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.751070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.751319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.751319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.751549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.751669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.751790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.751910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.752037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.752158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.752278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.752399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.752521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.752641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.752784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.752912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.753031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.753148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.753266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.753383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.753511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.753629 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.753747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.753864 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.753981 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.754098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.754223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.754342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.754468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.754588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.754706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.754830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.754947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.755064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.755181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.755298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.755415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.755539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.755657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.755774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.755891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.756017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.756135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.756261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.756379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.756496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.756645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.756780 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.756899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.757025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.757144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.757264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.757382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.757498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.757616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.757750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.757869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.757986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.758104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.758222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.758340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.758464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.758585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.758703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.758820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.758939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.759056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.759181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.759300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.759426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.759543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.759660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.759778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.759778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.760009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.760009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.760240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.760365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.760483 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.760609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.760779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.760901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.761032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.761164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.761285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.761405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.761524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.761644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.761772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.761893 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.762013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.762134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.762254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.762374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.762502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.762623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.762754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.762876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.762999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.763120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.763248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.763370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.763490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.763608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.763728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.763908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.764048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.764167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.764289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.764410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.764541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.764662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.764813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.764931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 10:21:37.765050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:21:37.765050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 10:21:37.790013 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.790177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.790343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.790468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.790592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.790679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.799108 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.799334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.827183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.827427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.827933 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.828227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.972135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:37.974874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.020636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.020879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.022216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.022414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.084360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.084615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.084791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.084879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.085011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.085093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.085217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.085295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.085424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.085503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.085629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.085708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.111025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.111229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.111445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.111690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.111878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.112119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.112288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.113059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.113257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.113515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.113704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.113924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.114167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.114342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.123900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.125391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.136502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.136884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.136987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.137170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.137261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.137386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.137465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.137588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.137678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.137796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.137912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.138029 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.138194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.138392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:21:38.155094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.155319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 10:21:38.155636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.155823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 10:21:38.156013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.156125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.156257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 10:21:38.156370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.156574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.156783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.156881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 10:21:38.157180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.157264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.157264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.157566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.157805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.167617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.167889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.168155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.168433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.168622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.168923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.169196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.169476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.176766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.178379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.180063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.180334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.180615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.180913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.181190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.181416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.181709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.181830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.181971 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.182131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.182255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.182392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.182508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.182629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.182749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.182870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.183003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.183290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.183561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.183834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.184107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.185206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.209889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.211506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.211782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.212297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.213950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.214230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.217075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.217558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.217838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.220069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.220464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.220768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.226133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.226133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.228601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.228918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.230298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.230585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.231725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.231725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.231725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.231725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.232220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.232220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.233996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.234282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.234570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.234850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.235126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.235404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.235677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.235945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.251353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.251629 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.255179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.255463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.256220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.259856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.260190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.260388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.263995 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.264903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.265321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.268940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.269967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.272585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.272908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.273202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.273469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.273591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.273735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.274024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.277319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.277615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.277900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.278181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.278458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.278742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.293290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.293573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.293837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.294109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.298922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.299210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.299477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.299753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.302238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.303943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.304197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.307632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.307869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.307966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.308166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.308451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.308759 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.309390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.309664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.313581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.313866 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.314146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.314423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.314714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.322052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.322331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.322616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.322895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.324525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.324847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.325134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.328903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.329185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.329478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.329762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.331391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.331690 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.331991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.343383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.346716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.350030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.350487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.351762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.353433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.353708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.353829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.354037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.357153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.359497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.361179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.361403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.361498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.361695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.363019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.366279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.366568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.366950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.367224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.369077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.370686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.374231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.374517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.374906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.375188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.376688 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.376997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.377285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.384350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.384639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.384943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.385127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.385407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.385682 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.385952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.386222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.386604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.386899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.387167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.387444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.387696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.387975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.388243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.388509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.388829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.393336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:21:38.393614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.393897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.394171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.394439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.394809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.395097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.395371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.395641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.395899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.396182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.396448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.396713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.397005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:21:38.426785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5636c58ec9c0 pid: 80146 (default app) Jun 24 10:21:38.430273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56647c4-3f5d-490a-bf5e-47af0875d191 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:38.430553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56647c4-3f5d-490a-bf5e-47af0875d191 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:38.430863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56647c4-3f5d-490a-bf5e-47af0875d191 None None] PATH_INFO: `/v3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:38.435096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1/1] 10.222.0.69 () {50 vars in 791 bytes} [Fri Jun 24 10:21:38 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:38.438194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5636c58ec9c0 pid: 80145 (default app) Jun 24 10:21:46.051052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe49fe-6eda-48bd-a72a-0f86469cc096 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:46.051981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe49fe-6eda-48bd-a72a-0f86469cc096 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:46.051981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe49fe-6eda-48bd-a72a-0f86469cc096 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:46.054135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 2/2] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:21:46.064565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:46.065057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:46.065146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:46.133305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: INFO dbcounter [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] Registered counter for database keystone Jun 24 10:21:46.157738 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG oslo_db.sqlalchemy.engines [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80145) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 10:21:46.165804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writer thread running {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 10:21:46.365619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG passlib.handlers.bcrypt [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80145) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 10:21:46.365962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG passlib.handlers.bcrypt [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80145) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 10:21:46.423173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:46.424801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:46.561049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1/3] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 10:21:46.721753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe49fe-6eda-48bd-a72a-0f86469cc096 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:46.796753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: INFO dbcounter [None req-15fe49fe-6eda-48bd-a72a-0f86469cc096 None None] Registered counter for database keystone Jun 24 10:21:46.818308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG oslo_db.sqlalchemy.engines [None req-15fe49fe-6eda-48bd-a72a-0f86469cc096 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80146) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 10:21:46.826324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writer thread running {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 10:21:46.946202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe49fe-6eda-48bd-a72a-0f86469cc096 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:46.951592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd045a4-c070-4a75-920b-7fb1046b765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:46.952304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd045a4-c070-4a75-920b-7fb1046b765e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:46.952594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd045a4-c070-4a75-920b-7fb1046b765e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:46.952900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd045a4-c070-4a75-920b-7fb1046b765e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:46.962383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd045a4-c070-4a75-920b-7fb1046b765e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:47.013747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6fd045a4-c070-4a75-920b-7fb1046b765e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 10:21:47.014021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd045a4-c070-4a75-920b-7fb1046b765e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:47.024503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 3/4] 10.222.0.69 () {58 vars in 1324 bytes} [Fri Jun 24 10:21:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:21:47.031996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:47.052696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7a13d-6c43-4bba-a91c-470a032aa080 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:47.059070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923acfa-3667-4716-b9f1-b0a3bd30c3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:47.060212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923acfa-3667-4716-b9f1-b0a3bd30c3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:47.060496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923acfa-3667-4716-b9f1-b0a3bd30c3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:47.060834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923acfa-3667-4716-b9f1-b0a3bd30c3fb admin admin] PATH_INFO: `/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:47.065343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923acfa-3667-4716-b9f1-b0a3bd30c3fb admin admin] RBAC: Authorizing `identity:get_project(project_id=0f9dd8794ad7494bbf369d7c8ce42c78)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:47.115527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d923acfa-3667-4716-b9f1-b0a3bd30c3fb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 10:21:47.115866 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923acfa-3667-4716-b9f1-b0a3bd30c3fb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:47.120457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 2/5] 10.222.0.69 () {56 vars in 1207 bytes} [Fri Jun 24 10:21:47 2022] GET /identity/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:49.093612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67301694-3755-4f90-b2d0-6bbd155acff4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.094427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67301694-3755-4f90-b2d0-6bbd155acff4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.094427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67301694-3755-4f90-b2d0-6bbd155acff4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.101207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 4/6] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:21:49.109529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aea3d6-d311-4b8f-8b16-ada1de06eb34 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.109981 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aea3d6-d311-4b8f-8b16-ada1de06eb34 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.110066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aea3d6-d311-4b8f-8b16-ada1de06eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.132049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-33aea3d6-d311-4b8f-8b16-ada1de06eb34 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:49.133630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-33aea3d6-d311-4b8f-8b16-ada1de06eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:49.144123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 3/7] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 24 10:21:49.166653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67301694-3755-4f90-b2d0-6bbd155acff4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:49.180961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67301694-3755-4f90-b2d0-6bbd155acff4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.186473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb93e42-24f5-4262-81e6-ab6b84e8f2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.187447 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb93e42-24f5-4262-81e6-ab6b84e8f2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.187736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb93e42-24f5-4262-81e6-ab6b84e8f2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.188020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb93e42-24f5-4262-81e6-ab6b84e8f2f5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.197930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb93e42-24f5-4262-81e6-ab6b84e8f2f5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.199726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb93e42-24f5-4262-81e6-ab6b84e8f2f5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.209726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 5/8] 10.222.0.69 () {58 vars in 1324 bytes} [Fri Jun 24 10:21:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:21:49.215764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aea3d6-d311-4b8f-8b16-ada1de06eb34 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:49.225553 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aea3d6-d311-4b8f-8b16-ada1de06eb34 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.233037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a823dc-74bb-4562-8558-8f8e015b71fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.233996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a823dc-74bb-4562-8558-8f8e015b71fa admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.234278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a823dc-74bb-4562-8558-8f8e015b71fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.234557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a823dc-74bb-4562-8558-8f8e015b71fa admin admin] PATH_INFO: `/v3/users/86fea4d6526441fa9d310035bfcc8012` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.236092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a823dc-74bb-4562-8558-8f8e015b71fa admin admin] RBAC: Authorizing `identity:get_user(user_id=86fea4d6526441fa9d310035bfcc8012)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.237976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a823dc-74bb-4562-8558-8f8e015b71fa admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.239520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 4/9] 10.222.0.69 () {56 vars in 1198 bytes} [Fri Jun 24 10:21:49 2022] GET /identity/v3/users/86fea4d6526441fa9d310035bfcc8012 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:51.320363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a8db2-2c15-4124-9fde-e5f837687e0a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.321157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a8db2-2c15-4124-9fde-e5f837687e0a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.321157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a8db2-2c15-4124-9fde-e5f837687e0a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.322781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 6/10] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:21:51.323389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e10bd-5ca7-429e-99b6-3d6774f53ae5 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.323783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e10bd-5ca7-429e-99b6-3d6774f53ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.323958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e10bd-5ca7-429e-99b6-3d6774f53ae5 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.325324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 5/11] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:21:51.327968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638fa01-c560-4d42-be40-4b616bc7665b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.328257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638fa01-c560-4d42-be40-4b616bc7665b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.328534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638fa01-c560-4d42-be40-4b616bc7665b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.330797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10457d88-b81e-49e6-904b-a47d67974f31 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.331089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10457d88-b81e-49e6-904b-a47d67974f31 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.331372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10457d88-b81e-49e6-904b-a47d67974f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.347613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-10457d88-b81e-49e6-904b-a47d67974f31 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:51.348810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-10457d88-b81e-49e6-904b-a47d67974f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:51.368277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG passlib.handlers.bcrypt [None req-1638fa01-c560-4d42-be40-4b616bc7665b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80146) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 10:21:51.368700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG passlib.handlers.bcrypt [None req-1638fa01-c560-4d42-be40-4b616bc7665b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80146) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 10:21:51.413698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1638fa01-c560-4d42-be40-4b616bc7665b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:51.415282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1638fa01-c560-4d42-be40-4b616bc7665b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:51.430262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 6/12] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:21:51.453942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10457d88-b81e-49e6-904b-a47d67974f31 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:51.477604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:51.517734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 7/13] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:21:51.540087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638fa01-c560-4d42-be40-4b616bc7665b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:51.558456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10457d88-b81e-49e6-904b-a47d67974f31 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:51.593585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f07bd8-ad15-49ec-a003-a51ab690dc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:51.594282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f07bd8-ad15-49ec-a003-a51ab690dc92 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.594567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f07bd8-ad15-49ec-a003-a51ab690dc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.594847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f07bd8-ad15-49ec-a003-a51ab690dc92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.606005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f07bd8-ad15-49ec-a003-a51ab690dc92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-12f07bd8-ad15-49ec-a003-a51ab690dc92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:21:51.617198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:21:51.618153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:21:51.618966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 7/14] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:21:51.624196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f07bd8-ad15-49ec-a003-a51ab690dc92 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:51.644328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638fa01-c560-4d42-be40-4b616bc7665b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:51.646687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:51.683450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f12da-28c2-4589-88ed-e8f7850b99ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:51.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f12da-28c2-4589-88ed-e8f7850b99ec None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.684762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f12da-28c2-4589-88ed-e8f7850b99ec None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.685046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f12da-28c2-4589-88ed-e8f7850b99ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.705182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:51.723442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f07bd8-ad15-49ec-a003-a51ab690dc92 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:51.752826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f12da-28c2-4589-88ed-e8f7850b99ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:51.754366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f12da-28c2-4589-88ed-e8f7850b99ec None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:51.755709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55297559-9e09-4d9e-9c60-c4e26bc11ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:51.756373 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297559-9e09-4d9e-9c60-c4e26bc11ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.756656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297559-9e09-4d9e-9c60-c4e26bc11ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.756964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297559-9e09-4d9e-9c60-c4e26bc11ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.757716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55297559-9e09-4d9e-9c60-c4e26bc11ba9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:51.758025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55297559-9e09-4d9e-9c60-c4e26bc11ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:51.759404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55297559-9e09-4d9e-9c60-c4e26bc11ba9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:51.771275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 8/15] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:21:51.776684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55297559-9e09-4d9e-9c60-c4e26bc11ba9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:51.851534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 8/16] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:21:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:21:51.857364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f12da-28c2-4589-88ed-e8f7850b99ec None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:51.860962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:51.880361 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55297559-9e09-4d9e-9c60-c4e26bc11ba9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:51.916403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8473e3-9a76-48ea-b668-f1ee82c8975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:51.917340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8473e3-9a76-48ea-b668-f1ee82c8975c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.917625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8473e3-9a76-48ea-b668-f1ee82c8975c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.917901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8473e3-9a76-48ea-b668-f1ee82c8975c None admin] PATH_INFO: `/v3/users/86fea4d6526441fa9d310035bfcc8012` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.919120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8473e3-9a76-48ea-b668-f1ee82c8975c None admin] RBAC: Authorizing `identity:get_user(user_id=86fea4d6526441fa9d310035bfcc8012)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:51.920663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8473e3-9a76-48ea-b668-f1ee82c8975c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:51.922460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 9/17] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:21:51 2022] GET /identity/v3/users/86fea4d6526441fa9d310035bfcc8012 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:21:51.927778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8473e3-9a76-48ea-b668-f1ee82c8975c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:51.942169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:51.958709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f12da-28c2-4589-88ed-e8f7850b99ec None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:51.992422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21509ff-9d9d-4048-a7c5-e76da2aa4d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:51.993262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21509ff-9d9d-4048-a7c5-e76da2aa4d87 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.993547 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21509ff-9d9d-4048-a7c5-e76da2aa4d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.993837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21509ff-9d9d-4048-a7c5-e76da2aa4d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.012449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21509ff-9d9d-4048-a7c5-e76da2aa4d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:52.014042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21509ff-9d9d-4048-a7c5-e76da2aa4d87 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:52.026977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8473e3-9a76-48ea-b668-f1ee82c8975c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f21509ff-9d9d-4048-a7c5-e76da2aa4d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:21:52.036180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:21:52.037028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:21:52.037815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:21:52.037815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:21:52.037815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:21:52.037815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:21:52.037815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 9/18] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:21:52.043303 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21509ff-9d9d-4048-a7c5-e76da2aa4d87 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:52.062966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ece3a-a74f-4532-b9fe-f93d3ecd1464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:52.063761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ece3a-a74f-4532-b9fe-f93d3ecd1464 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.064044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ece3a-a74f-4532-b9fe-f93d3ecd1464 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.064320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ece3a-a74f-4532-b9fe-f93d3ecd1464 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.065550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ece3a-a74f-4532-b9fe-f93d3ecd1464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:52.067009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ece3a-a74f-4532-b9fe-f93d3ecd1464 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:52.068632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 10/19] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:21:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:21:52.074263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ece3a-a74f-4532-b9fe-f93d3ecd1464 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:52.100065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:52.145524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21509ff-9d9d-4048-a7c5-e76da2aa4d87 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:52.156596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:52.176666 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ece3a-a74f-4532-b9fe-f93d3ecd1464 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:52.179448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21983c1a-0f22-4372-8824-539e724cf814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:52.180115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21983c1a-0f22-4372-8824-539e724cf814 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.180398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21983c1a-0f22-4372-8824-539e724cf814 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.180727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21983c1a-0f22-4372-8824-539e724cf814 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.181518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21983c1a-0f22-4372-8824-539e724cf814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:52.181825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21983c1a-0f22-4372-8824-539e724cf814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:52.183214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21983c1a-0f22-4372-8824-539e724cf814 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:52.200424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 10/20] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:21:52.212210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4386b5-18c5-4be8-b9e8-8e3fe39ede1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:52.212929 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4386b5-18c5-4be8-b9e8-8e3fe39ede1a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.213209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4386b5-18c5-4be8-b9e8-8e3fe39ede1a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.213481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4386b5-18c5-4be8-b9e8-8e3fe39ede1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.214333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4386b5-18c5-4be8-b9e8-8e3fe39ede1a None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.domain.id=default, user.id=86fea4d6526441fa9d310035bfcc8012) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:52.214644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4386b5-18c5-4be8-b9e8-8e3fe39ede1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:52.216130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4386b5-18c5-4be8-b9e8-8e3fe39ede1a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:52.224204 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 11/21] 10.222.0.69 () {56 vars in 1322 bytes} [Fri Jun 24 10:21:52 2022] GET /identity/v3/role_assignments?user.id=86fea4d6526441fa9d310035bfcc8012&scope.domain.id=default&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:21:54.605780 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3085e7-9d02-4e55-b533-a6faf37be033 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.606278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3085e7-9d02-4e55-b533-a6faf37be033 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.606348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3085e7-9d02-4e55-b533-a6faf37be033 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.607935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 11/22] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:21:54.614354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993a34e-2a22-4604-b505-d1cfa7e106cc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.615175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993a34e-2a22-4604-b505-d1cfa7e106cc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.615639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993a34e-2a22-4604-b505-d1cfa7e106cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.636626 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8993a34e-2a22-4604-b505-d1cfa7e106cc None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:54.638186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8993a34e-2a22-4604-b505-d1cfa7e106cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:54.734331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 12/23] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:21:54.768995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3085e7-9d02-4e55-b533-a6faf37be033 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:54.891332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71375a2b-76f3-4120-92d0-791a4a4dbbc3 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.891950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71375a2b-76f3-4120-92d0-791a4a4dbbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.892463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71375a2b-76f3-4120-92d0-791a4a4dbbc3 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.894056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 13/24] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:21:54.899563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e34f-fc40-4f09-900f-5fab4b4a37a0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.900071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e34f-fc40-4f09-900f-5fab4b4a37a0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.900531 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e34f-fc40-4f09-900f-5fab4b4a37a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.909542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3085e7-9d02-4e55-b533-a6faf37be033 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:54.937935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f6a1e34f-fc40-4f09-900f-5fab4b4a37a0 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:54.939446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f6a1e34f-fc40-4f09-900f-5fab4b4a37a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:54.956783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:54.971613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd659069-ea29-4b4d-8f76-e0244a0acf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:54.973097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd659069-ea29-4b4d-8f76-e0244a0acf8e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.973586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd659069-ea29-4b4d-8f76-e0244a0acf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.974045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd659069-ea29-4b4d-8f76-e0244a0acf8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.987843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd659069-ea29-4b4d-8f76-e0244a0acf8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:54.989634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd659069-ea29-4b4d-8f76-e0244a0acf8e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-bd659069-ea29-4b4d-8f76-e0244a0acf8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:21:55.009109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:21:55.009923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:21:55.010690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:21:55.010690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:21:55.010690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:21:55.010690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:21:55.012607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 12/25] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:21:55.014399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2b13d-4353-49fa-857a-b99a18ad50c1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.014897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2b13d-4353-49fa-857a-b99a18ad50c1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.015356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2b13d-4353-49fa-857a-b99a18ad50c1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.016831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 13/26] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:21:55.021851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2b13d-4353-49fa-857a-b99a18ad50c1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:55.089987 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 14/27] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:21:55.091571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cf4e4-e259-4158-b5ab-0e7ee2a2691e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.092028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cf4e4-e259-4158-b5ab-0e7ee2a2691e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.092472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cf4e4-e259-4158-b5ab-0e7ee2a2691e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.098558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:55.110707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ee2cf4e4-e259-4158-b5ab-0e7ee2a2691e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:55.112210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ee2cf4e4-e259-4158-b5ab-0e7ee2a2691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:55.154290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2b13d-4353-49fa-857a-b99a18ad50c1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:55.182110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 15/28] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:21:55.184032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608808b-e400-4d94-83c3-9e8c68b9de5a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.184519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608808b-e400-4d94-83c3-9e8c68b9de5a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.185027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608808b-e400-4d94-83c3-9e8c68b9de5a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.186832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 16/29] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:21:55.188882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f2136-c8b3-4341-b8a6-0bcdf353ed45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:55.189706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f2136-c8b3-4341-b8a6-0bcdf353ed45 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.190132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f2136-c8b3-4341-b8a6-0bcdf353ed45 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.190524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f2136-c8b3-4341-b8a6-0bcdf353ed45 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.191644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f2136-c8b3-4341-b8a6-0bcdf353ed45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:55.191991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608808b-e400-4d94-83c3-9e8c68b9de5a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:55.192232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f2136-c8b3-4341-b8a6-0bcdf353ed45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:55.193773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f2136-c8b3-4341-b8a6-0bcdf353ed45 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:55.206375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 14/30] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:55.207817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1556b-fae4-495b-9df9-1dc9ee4c009c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.208227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1556b-fae4-495b-9df9-1dc9ee4c009c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.208618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1556b-fae4-495b-9df9-1dc9ee4c009c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.228505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f0c1556b-fae4-495b-9df9-1dc9ee4c009c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:55.230467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f0c1556b-fae4-495b-9df9-1dc9ee4c009c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:55.256328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:55.307547 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 15/31] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:21:55.308641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1556b-fae4-495b-9df9-1dc9ee4c009c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:55.347986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608808b-e400-4d94-83c3-9e8c68b9de5a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:55.394514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee1acd-bbb1-431e-85f6-27da53d34460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:55.395444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1acd-bbb1-431e-85f6-27da53d34460 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.395734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1acd-bbb1-431e-85f6-27da53d34460 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.396014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1acd-bbb1-431e-85f6-27da53d34460 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.396783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee1acd-bbb1-431e-85f6-27da53d34460 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:55.399304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee1acd-bbb1-431e-85f6-27da53d34460 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:55.401798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:55.432252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1556b-fae4-495b-9df9-1dc9ee4c009c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:55.469675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d2bee-ac00-4d4c-a7a0-59a60b272ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:55.470557 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d2bee-ac00-4d4c-a7a0-59a60b272ccc None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.470982 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d2bee-ac00-4d4c-a7a0-59a60b272ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.471420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d2bee-ac00-4d4c-a7a0-59a60b272ccc None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.472554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d2bee-ac00-4d4c-a7a0-59a60b272ccc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:55.474174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d2bee-ac00-4d4c-a7a0-59a60b272ccc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:55.534671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 17/32] 10.222.0.69 () {60 vars in 1131 bytes} [Fri Jun 24 10:21:55 2022] POST /identity/v3/roles => generated 225 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:55.535928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee1acd-bbb1-431e-85f6-27da53d34460 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:55.617634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 16/33] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:21:55 2022] POST /identity/v3/projects => generated 301 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:55.619284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d2bee-ac00-4d4c-a7a0-59a60b272ccc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:55.635108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:55.657962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee1acd-bbb1-431e-85f6-27da53d34460 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:55.700722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74c12c-51ad-4b78-a75b-85cadd01e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:55.701810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74c12c-51ad-4b78-a75b-85cadd01e5e6 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.702258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74c12c-51ad-4b78-a75b-85cadd01e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.702670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74c12c-51ad-4b78-a75b-85cadd01e5e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.703572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74c12c-51ad-4b78-a75b-85cadd01e5e6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:55.705800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74c12c-51ad-4b78-a75b-85cadd01e5e6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:55.732721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 18/34] 10.222.0.69 () {60 vars in 1131 bytes} [Fri Jun 24 10:21:55 2022] POST /identity/v3/roles => generated 219 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:55.746747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d2bee-ac00-4d4c-a7a0-59a60b272ccc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:55.791409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc5536-138a-4f50-ab13-a8226f6b8621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:55.792308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc5536-138a-4f50-ab13-a8226f6b8621 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.792608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc5536-138a-4f50-ab13-a8226f6b8621 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.792921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc5536-138a-4f50-ab13-a8226f6b8621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.807850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc5536-138a-4f50-ab13-a8226f6b8621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-afdc5536-138a-4f50-ab13-a8226f6b8621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:21:55.817594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:21:55.818505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:21:55.819234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 17/35] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:21:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:21:55.826534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74c12c-51ad-4b78-a75b-85cadd01e5e6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:55.897018 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:55.950399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74c12c-51ad-4b78-a75b-85cadd01e5e6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:55.991049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b49711-fb57-43e6-a7e4-f04811936833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:55.991741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b49711-fb57-43e6-a7e4-f04811936833 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:55.992031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b49711-fb57-43e6-a7e4-f04811936833 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:55.992327 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b49711-fb57-43e6-a7e4-f04811936833 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:55.993141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b49711-fb57-43e6-a7e4-f04811936833 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:55.993501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b49711-fb57-43e6-a7e4-f04811936833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:55.994923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b49711-fb57-43e6-a7e4-f04811936833 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:56.004730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 19/36] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:21:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:56.010501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc5536-138a-4f50-ab13-a8226f6b8621 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:56.051016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea29e76-10e0-4eb0-baec-a3575abfb407 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.051258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea29e76-10e0-4eb0-baec-a3575abfb407 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.051514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea29e76-10e0-4eb0-baec-a3575abfb407 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.052956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 20/37] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:21:56.058193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182289e-ca39-47fd-9b30-652b9697e708 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.058478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182289e-ca39-47fd-9b30-652b9697e708 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.058760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182289e-ca39-47fd-9b30-652b9697e708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.079436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a182289e-ca39-47fd-9b30-652b9697e708 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:56.082472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a182289e-ca39-47fd-9b30-652b9697e708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:56.097561 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:56.139210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc5536-138a-4f50-ab13-a8226f6b8621 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:56.160048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 21/38] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:21:56.179928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99bc4-8286-4ae5-825f-9bc429c30255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:56.180851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99bc4-8286-4ae5-825f-9bc429c30255 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.181147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99bc4-8286-4ae5-825f-9bc429c30255 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.181431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99bc4-8286-4ae5-825f-9bc429c30255 None admin] PATH_INFO: `/v3/users/86fea4d6526441fa9d310035bfcc8012` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.183262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182289e-ca39-47fd-9b30-652b9697e708 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:56.185891 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf99bc4-8286-4ae5-825f-9bc429c30255 None admin] RBAC: Authorizing `identity:get_user(user_id=86fea4d6526441fa9d310035bfcc8012)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:56.187677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf99bc4-8286-4ae5-825f-9bc429c30255 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:56.189584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 18/39] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:21:56 2022] GET /identity/v3/users/86fea4d6526441fa9d310035bfcc8012 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:56.194991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99bc4-8286-4ae5-825f-9bc429c30255 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:56.232666 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:56.286023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182289e-ca39-47fd-9b30-652b9697e708 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:56.295961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99bc4-8286-4ae5-825f-9bc429c30255 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:56.320324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7cb57-5517-4105-a017-949d619358db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:56.321061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7cb57-5517-4105-a017-949d619358db None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.321341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7cb57-5517-4105-a017-949d619358db None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.321614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7cb57-5517-4105-a017-949d619358db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.322952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7cb57-5517-4105-a017-949d619358db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:56.324445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7cb57-5517-4105-a017-949d619358db None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:56.326307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 22/40] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:21:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:21:56.327310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec079e5-de38-4d24-bb25-beacddc91a6b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.327602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec079e5-de38-4d24-bb25-beacddc91a6b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.327881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec079e5-de38-4d24-bb25-beacddc91a6b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.329493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 23/41] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:21:56.332144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec079e5-de38-4d24-bb25-beacddc91a6b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:56.337631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795340a-54c6-4450-8072-dbbf8578281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:56.338442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795340a-54c6-4450-8072-dbbf8578281d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.338731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795340a-54c6-4450-8072-dbbf8578281d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.339012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795340a-54c6-4450-8072-dbbf8578281d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.340300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795340a-54c6-4450-8072-dbbf8578281d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:56.341793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795340a-54c6-4450-8072-dbbf8578281d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:56.343766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 19/42] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:21:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:21:56.345028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aeb8a-9c94-4f19-9dc9-09a64acc3016 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.345324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aeb8a-9c94-4f19-9dc9-09a64acc3016 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.345598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aeb8a-9c94-4f19-9dc9-09a64acc3016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.358313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:56.364392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-932aeb8a-9c94-4f19-9dc9-09a64acc3016 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:56.365961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-932aeb8a-9c94-4f19-9dc9-09a64acc3016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:56.432525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 20/43] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:21:56.432690 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec079e5-de38-4d24-bb25-beacddc91a6b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:56.433342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aeb8a-9c94-4f19-9dc9-09a64acc3016 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:56.457860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:56.478370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65913f-7b11-4bd6-b1bf-50e41985cb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:56.479042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65913f-7b11-4bd6-b1bf-50e41985cb2b None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.479356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65913f-7b11-4bd6-b1bf-50e41985cb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.479641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65913f-7b11-4bd6-b1bf-50e41985cb2b None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.480359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65913f-7b11-4bd6-b1bf-50e41985cb2b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:56.481861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65913f-7b11-4bd6-b1bf-50e41985cb2b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:56.509757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 24/44] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:21:56 2022] POST /identity/v3/projects => generated 312 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:21:56.510552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65913f-7b11-4bd6-b1bf-50e41985cb2b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:56.542702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aeb8a-9c94-4f19-9dc9-09a64acc3016 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:56.586906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb52b2f-285b-4185-a3fe-ceadf698143b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:56.587796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb52b2f-285b-4185-a3fe-ceadf698143b None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.588081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb52b2f-285b-4185-a3fe-ceadf698143b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.588399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb52b2f-285b-4185-a3fe-ceadf698143b None admin] PATH_INFO: `/v3/domains/default/users/86fea4d6526441fa9d310035bfcc8012/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.590684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb52b2f-285b-4185-a3fe-ceadf698143b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=86fea4d6526441fa9d310035bfcc8012, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:56.592318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb52b2f-285b-4185-a3fe-ceadf698143b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:56.598164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:56.616039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 21/45] 10.222.0.69 () {58 vars in 1361 bytes} [Fri Jun 24 10:21:56 2022] PUT /identity/v3/domains/default/users/86fea4d6526441fa9d310035bfcc8012/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:56.742318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65913f-7b11-4bd6-b1bf-50e41985cb2b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:56.781432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259340cb-76f5-4ffc-a250-b183483027f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:56.782118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259340cb-76f5-4ffc-a250-b183483027f9 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.782401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259340cb-76f5-4ffc-a250-b183483027f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.782679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259340cb-76f5-4ffc-a250-b183483027f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.783437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259340cb-76f5-4ffc-a250-b183483027f9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:56.784910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259340cb-76f5-4ffc-a250-b183483027f9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:56.801590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 25/46] 10.222.0.69 () {60 vars in 1131 bytes} [Fri Jun 24 10:21:56 2022] POST /identity/v3/roles => generated 223 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:58.551217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518b1bd-64a9-4996-a320-676121b72646 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:58.551745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518b1bd-64a9-4996-a320-676121b72646 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:58.551745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518b1bd-64a9-4996-a320-676121b72646 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:58.553205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 22/47] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:21:58.558993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb2cf3-dda3-4d84-a3fa-d817cf6b7ed0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:58.559422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb2cf3-dda3-4d84-a3fa-d817cf6b7ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:58.559616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb2cf3-dda3-4d84-a3fa-d817cf6b7ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:58.577642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-40cb2cf3-dda3-4d84-a3fa-d817cf6b7ed0 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:58.579135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-40cb2cf3-dda3-4d84-a3fa-d817cf6b7ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:58.601152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:58.610636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447415b6-8558-4be8-919e-b3401ec506e2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:58.610957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447415b6-8558-4be8-919e-b3401ec506e2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:58.611232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447415b6-8558-4be8-919e-b3401ec506e2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:58.612584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 23/48] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:21:58.618226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89896ab-1e36-4c81-940d-4e3a0548d27c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:58.618657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89896ab-1e36-4c81-940d-4e3a0548d27c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:58.619093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89896ab-1e36-4c81-940d-4e3a0548d27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:58.637500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e89896ab-1e36-4c81-940d-4e3a0548d27c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:58.638704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e89896ab-1e36-4c81-940d-4e3a0548d27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:58.652788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 26/49] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:21:58.677178 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb2cf3-dda3-4d84-a3fa-d817cf6b7ed0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:58.704583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 24/50] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:21:58.728170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89896ab-1e36-4c81-940d-4e3a0548d27c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:58.785710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb2cf3-dda3-4d84-a3fa-d817cf6b7ed0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:58.819917 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d190f54-6729-44ee-a79d-7ffaf9d1798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:58.820718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190f54-6729-44ee-a79d-7ffaf9d1798e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:58.821036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190f54-6729-44ee-a79d-7ffaf9d1798e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:58.821322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190f54-6729-44ee-a79d-7ffaf9d1798e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:58.822512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d190f54-6729-44ee-a79d-7ffaf9d1798e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:58.823994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d190f54-6729-44ee-a79d-7ffaf9d1798e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:58.825719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 27/51] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:21:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:58.831564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d190f54-6729-44ee-a79d-7ffaf9d1798e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:58.836519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89896ab-1e36-4c81-940d-4e3a0548d27c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:58.856442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:58.863401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:58.873347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5bf9a-43f4-48d0-850f-441a2adeb2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:58.874206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5bf9a-43f4-48d0-850f-441a2adeb2fa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:58.874495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5bf9a-43f4-48d0-850f-441a2adeb2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:58.874774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5bf9a-43f4-48d0-850f-441a2adeb2fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:58.885876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5bf9a-43f4-48d0-850f-441a2adeb2fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b8c5bf9a-43f4-48d0-850f-441a2adeb2fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:21:58.893636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:21:58.894444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:21:58.895175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 25/52] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:21:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:21:58.900556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5bf9a-43f4-48d0-850f-441a2adeb2fa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:58.933003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d190f54-6729-44ee-a79d-7ffaf9d1798e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:58.965806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e31b8e-8b74-477b-9976-d41582f58e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:58.966470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e31b8e-8b74-477b-9976-d41582f58e9e None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:58.966761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e31b8e-8b74-477b-9976-d41582f58e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:58.967039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e31b8e-8b74-477b-9976-d41582f58e9e None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:58.967823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e31b8e-8b74-477b-9976-d41582f58e9e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:58.969343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e31b8e-8b74-477b-9976-d41582f58e9e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:58.995766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 28/53] 10.222.0.69 () {60 vars in 1140 bytes} [Fri Jun 24 10:21:58 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:21:59.002845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5bf9a-43f4-48d0-850f-441a2adeb2fa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:59.041340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdae6b-a079-4c51-b053-8630aa67cf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:59.042026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdae6b-a079-4c51-b053-8630aa67cf53 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:59.042310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdae6b-a079-4c51-b053-8630aa67cf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:59.042590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdae6b-a079-4c51-b053-8630aa67cf53 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:59.043376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdae6b-a079-4c51-b053-8630aa67cf53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:59.043687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdae6b-a079-4c51-b053-8630aa67cf53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:59.045167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdae6b-a079-4c51-b053-8630aa67cf53 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:59.168209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 26/54] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:21:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:59.174011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e31b8e-8b74-477b-9976-d41582f58e9e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:59.258906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:59.284016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e31b8e-8b74-477b-9976-d41582f58e9e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:59.319120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f026aee-bc89-495d-8a5f-b747e90f2810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:59.319998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f026aee-bc89-495d-8a5f-b747e90f2810 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:59.320282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f026aee-bc89-495d-8a5f-b747e90f2810 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:59.320561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f026aee-bc89-495d-8a5f-b747e90f2810 None admin] PATH_INFO: `/v3/users/86fea4d6526441fa9d310035bfcc8012` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:59.321830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f026aee-bc89-495d-8a5f-b747e90f2810 None admin] RBAC: Authorizing `identity:get_user(user_id=86fea4d6526441fa9d310035bfcc8012)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:59.323285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f026aee-bc89-495d-8a5f-b747e90f2810 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:59.325165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 29/55] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:21:59 2022] GET /identity/v3/users/86fea4d6526441fa9d310035bfcc8012 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:59.330987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdae6b-a079-4c51-b053-8630aa67cf53 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:59.383340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:59.437099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdae6b-a079-4c51-b053-8630aa67cf53 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:59.475946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910dcb6-71fe-408f-b634-57841238950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:59.476768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910dcb6-71fe-408f-b634-57841238950c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:59.477071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910dcb6-71fe-408f-b634-57841238950c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:59.477347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910dcb6-71fe-408f-b634-57841238950c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:59.478556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910dcb6-71fe-408f-b634-57841238950c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:59.480023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910dcb6-71fe-408f-b634-57841238950c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:59.481685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 27/56] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:21:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:59.488126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f026aee-bc89-495d-8a5f-b747e90f2810 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:59.593118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f026aee-bc89-495d-8a5f-b747e90f2810 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:59.626623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c188e-211f-41de-a861-7087590ecb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:59.627273 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c188e-211f-41de-a861-7087590ecb1f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:59.627556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c188e-211f-41de-a861-7087590ecb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:59.627833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c188e-211f-41de-a861-7087590ecb1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:59.628669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c188e-211f-41de-a861-7087590ecb1f None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.domain.id=default, user.id=86fea4d6526441fa9d310035bfcc8012) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:59.629007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c188e-211f-41de-a861-7087590ecb1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:59.630456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c188e-211f-41de-a861-7087590ecb1f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:59.636457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 30/57] 10.222.0.69 () {56 vars in 1322 bytes} [Fri Jun 24 10:21:59 2022] GET /identity/v3/role_assignments?user.id=86fea4d6526441fa9d310035bfcc8012&scope.domain.id=default&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:01.053082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f13360-0bd5-462e-b9d1-35a7e5ea3c49 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:01.053873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f13360-0bd5-462e-b9d1-35a7e5ea3c49 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:01.053977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f13360-0bd5-462e-b9d1-35a7e5ea3c49 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:01.056117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 28/58] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:01.062797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7ca16-93d6-4581-9064-83bd3f07129a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:01.063447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7ca16-93d6-4581-9064-83bd3f07129a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:01.063732 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7ca16-93d6-4581-9064-83bd3f07129a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:01.087045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e5a7ca16-93d6-4581-9064-83bd3f07129a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:01.088245 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e5a7ca16-93d6-4581-9064-83bd3f07129a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:01.090352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:01.160769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 31/59] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:01.185721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f13360-0bd5-462e-b9d1-35a7e5ea3c49 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:01.291353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f13360-0bd5-462e-b9d1-35a7e5ea3c49 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:01.308198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:01.327249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48ea50-d7b0-400d-808a-bba6ca89a8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:01.328037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48ea50-d7b0-400d-808a-bba6ca89a8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:01.328322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48ea50-d7b0-400d-808a-bba6ca89a8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:01.328591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48ea50-d7b0-400d-808a-bba6ca89a8c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:01.330157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48ea50-d7b0-400d-808a-bba6ca89a8c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:01.331608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48ea50-d7b0-400d-808a-bba6ca89a8c3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:01.333250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 29/60] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:01.339007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7ca16-93d6-4581-9064-83bd3f07129a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:01.453165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7ca16-93d6-4581-9064-83bd3f07129a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:01.490398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6ecd2-83a6-47dd-968d-38407a6d3b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:01.491066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6ecd2-83a6-47dd-968d-38407a6d3b4e None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:01.491348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6ecd2-83a6-47dd-968d-38407a6d3b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:01.491621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6ecd2-83a6-47dd-968d-38407a6d3b4e None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:01.492326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6ecd2-83a6-47dd-968d-38407a6d3b4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:01.493792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6ecd2-83a6-47dd-968d-38407a6d3b4e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:01.507445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:01.525699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 32/61] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:01 2022] POST /identity/v3/users => generated 275 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:04.609237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fb20b-12b7-4776-bd42-ab3058f1c917 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:04.610279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fb20b-12b7-4776-bd42-ab3058f1c917 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:04.610788 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fb20b-12b7-4776-bd42-ab3058f1c917 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:04.612414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 30/62] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:04.618292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533f437-ab19-4cc8-a7d6-03b489eaa052 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:04.619319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533f437-ab19-4cc8-a7d6-03b489eaa052 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:04.619825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533f437-ab19-4cc8-a7d6-03b489eaa052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:04.644056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8533f437-ab19-4cc8-a7d6-03b489eaa052 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:04.646397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8533f437-ab19-4cc8-a7d6-03b489eaa052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:04.734118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 33/63] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:04.768340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fb20b-12b7-4776-bd42-ab3058f1c917 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:04.918458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fb20b-12b7-4776-bd42-ab3058f1c917 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:04.965559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3360f-a538-4bf5-96f8-a149f3b65e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:04.966414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3360f-a538-4bf5-96f8-a149f3b65e51 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:04.966705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3360f-a538-4bf5-96f8-a149f3b65e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:04.966987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3360f-a538-4bf5-96f8-a149f3b65e51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:04.968511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667d154-fee9-4db7-aecb-ae0f248c86a0 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:04.969051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667d154-fee9-4db7-aecb-ae0f248c86a0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:04.969511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667d154-fee9-4db7-aecb-ae0f248c86a0 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:04.971163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 34/64] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:04.977177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a0c06-e649-47da-8413-93294503c525 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:04.977724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a0c06-e649-47da-8413-93294503c525 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:04.978226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a0c06-e649-47da-8413-93294503c525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:04.982647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3360f-a538-4bf5-96f8-a149f3b65e51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6ca3360f-a538-4bf5-96f8-a149f3b65e51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:04.994278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:04.995144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:04.995882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:04.998226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 31/65] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:05.005147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3360f-a538-4bf5-96f8-a149f3b65e51 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.012094 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-003a0c06-e649-47da-8413-93294503c525 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:05.014474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-003a0c06-e649-47da-8413-93294503c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:05.073660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:05.150589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3360f-a538-4bf5-96f8-a149f3b65e51 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:05.151653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 35/66] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:05.154470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef808643-3665-4d51-b3ae-c426257af793 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.157909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef808643-3665-4d51-b3ae-c426257af793 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.158429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef808643-3665-4d51-b3ae-c426257af793 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.160054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 36/67] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:05.161589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8de11-badb-46ce-a3f9-fc8c5f0ae80d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.162099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8de11-badb-46ce-a3f9-fc8c5f0ae80d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.162848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8de11-badb-46ce-a3f9-fc8c5f0ae80d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.165640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 37/68] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:05.168985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1c80b-46d9-4adc-8baf-1368a6c56a07 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.169866 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1c80b-46d9-4adc-8baf-1368a6c56a07 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.176066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1c80b-46d9-4adc-8baf-1368a6c56a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.194168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3848f-00ec-449a-bf21-36c3bc951b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:05.194850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3848f-00ec-449a-bf21-36c3bc951b5e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.195132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3848f-00ec-449a-bf21-36c3bc951b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.195404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3848f-00ec-449a-bf21-36c3bc951b5e None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.196212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3848f-00ec-449a-bf21-36c3bc951b5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:05.196489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3848f-00ec-449a-bf21-36c3bc951b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:05.197250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4bb1c80b-46d9-4adc-8baf-1368a6c56a07 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:05.197891 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3848f-00ec-449a-bf21-36c3bc951b5e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:05.198702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4bb1c80b-46d9-4adc-8baf-1368a6c56a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:05.212554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 32/69] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:05.212554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62512d-d979-4df1-bdc1-f6a8e9a481d8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.212554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62512d-d979-4df1-bdc1-f6a8e9a481d8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.212554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62512d-d979-4df1-bdc1-f6a8e9a481d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.228190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5b62512d-d979-4df1-bdc1-f6a8e9a481d8 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:05.229412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5b62512d-d979-4df1-bdc1-f6a8e9a481d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:05.271832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:05.294173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 38/70] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:05.295520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1c80b-46d9-4adc-8baf-1368a6c56a07 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.306769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 33/71] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:05.307855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62512d-d979-4df1-bdc1-f6a8e9a481d8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.356088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:05.423042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62512d-d979-4df1-bdc1-f6a8e9a481d8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:05.425570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1c80b-46d9-4adc-8baf-1368a6c56a07 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:05.459159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629c55e-baf8-4bbb-891b-de5ebd9753ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:05.460066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629c55e-baf8-4bbb-891b-de5ebd9753ca None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.460350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629c55e-baf8-4bbb-891b-de5ebd9753ca None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.460713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629c55e-baf8-4bbb-891b-de5ebd9753ca None admin] PATH_INFO: `/v3/users/c60d4a5f5d83459c99e38bd82d610465` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.466021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a9c37-bc3f-46c7-bfa3-060d96d73761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:05.466859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a9c37-bc3f-46c7-bfa3-060d96d73761 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.467143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a9c37-bc3f-46c7-bfa3-060d96d73761 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.467426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a9c37-bc3f-46c7-bfa3-060d96d73761 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.476390 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629c55e-baf8-4bbb-891b-de5ebd9753ca None admin] RBAC: Authorizing `identity:get_user(user_id=c60d4a5f5d83459c99e38bd82d610465)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:05.477930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629c55e-baf8-4bbb-891b-de5ebd9753ca None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:05.478475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a9c37-bc3f-46c7-bfa3-060d96d73761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:05.479679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 34/72] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:05 2022] GET /identity/v3/users/c60d4a5f5d83459c99e38bd82d610465 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:05.480513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629c55e-baf8-4bbb-891b-de5ebd9753ca None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ae3a9c37-bc3f-46c7-bfa3-060d96d73761 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:05.487432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:05.488255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:05.489043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 39/73] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:05.489636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a9c37-bc3f-46c7-bfa3-060d96d73761 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.538008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:05.581348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:05.588833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629c55e-baf8-4bbb-891b-de5ebd9753ca None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:05.595663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a9c37-bc3f-46c7-bfa3-060d96d73761 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:05.622589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e025-58c1-4aa2-b3e1-84b16d0bad85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:05.623397 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e025-58c1-4aa2-b3e1-84b16d0bad85 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.623819 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e025-58c1-4aa2-b3e1-84b16d0bad85 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.624125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e025-58c1-4aa2-b3e1-84b16d0bad85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.628452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e1840-ed10-4a54-b246-e1213efa22ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:05.629266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e1840-ed10-4a54-b246-e1213efa22ef None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.629547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e1840-ed10-4a54-b246-e1213efa22ef None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.629826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e1840-ed10-4a54-b246-e1213efa22ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.630988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e1840-ed10-4a54-b246-e1213efa22ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:05.632471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e1840-ed10-4a54-b246-e1213efa22ef None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:05.634120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 40/74] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:05.639064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e025-58c1-4aa2-b3e1-84b16d0bad85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:05.639166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62153f8-a6bb-4f62-acc2-d033d921f512 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.639166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62153f8-a6bb-4f62-acc2-d033d921f512 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.639166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62153f8-a6bb-4f62-acc2-d033d921f512 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.639166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 41/75] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:05.639166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62153f8-a6bb-4f62-acc2-d033d921f512 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a167e025-58c1-4aa2-b3e1-84b16d0bad85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:05.643227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:05.644043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:05.644866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 35/76] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:22:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:05.645844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e025-58c1-4aa2-b3e1-84b16d0bad85 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.741084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62153f8-a6bb-4f62-acc2-d033d921f512 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:05.746865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e025-58c1-4aa2-b3e1-84b16d0bad85 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:05.754305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:05.777152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036745c-3428-49b9-b81a-494ef9a02a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:05.777809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036745c-3428-49b9-b81a-494ef9a02a50 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.778096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036745c-3428-49b9-b81a-494ef9a02a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.778380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036745c-3428-49b9-b81a-494ef9a02a50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.780561 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93fef-a034-4ed7-af6d-30ab90f27714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:05.781270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93fef-a034-4ed7-af6d-30ab90f27714 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.781556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93fef-a034-4ed7-af6d-30ab90f27714 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.781811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93fef-a034-4ed7-af6d-30ab90f27714 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.782568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae93fef-a034-4ed7-af6d-30ab90f27714 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:05.782879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae93fef-a034-4ed7-af6d-30ab90f27714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:05.784259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae93fef-a034-4ed7-af6d-30ab90f27714 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:05.792367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 36/77] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:05.793101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93fef-a034-4ed7-af6d-30ab90f27714 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.810924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:05.848938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036745c-3428-49b9-b81a-494ef9a02a50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:05.850421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036745c-3428-49b9-b81a-494ef9a02a50 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:05.894374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93fef-a034-4ed7-af6d-30ab90f27714 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:05.927839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69726bc-ec35-4817-a8b5-fd8b64a3dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:05.928514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69726bc-ec35-4817-a8b5-fd8b64a3dd45 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.928821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69726bc-ec35-4817-a8b5-fd8b64a3dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.929105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69726bc-ec35-4817-a8b5-fd8b64a3dd45 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.929821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69726bc-ec35-4817-a8b5-fd8b64a3dd45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:05.931266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69726bc-ec35-4817-a8b5-fd8b64a3dd45 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:05.947504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 42/78] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:05.948905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71913b0-ebaf-47b7-9e8a-27530da0fdc2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.949262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71913b0-ebaf-47b7-9e8a-27530da0fdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.949620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71913b0-ebaf-47b7-9e8a-27530da0fdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.976763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 37/79] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:05 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:05.976763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69726bc-ec35-4817-a8b5-fd8b64a3dd45 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.977047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a71913b0-ebaf-47b7-9e8a-27530da0fdc2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:05.977047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a71913b0-ebaf-47b7-9e8a-27530da0fdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:05.977171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:06.045821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 43/80] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:06.046699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71913b0-ebaf-47b7-9e8a-27530da0fdc2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:06.063231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:06.099739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69726bc-ec35-4817-a8b5-fd8b64a3dd45 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:06.139120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135ac9-8a9b-408d-ad2a-116e1c480a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:06.139969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135ac9-8a9b-408d-ad2a-116e1c480a15 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:06.140250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135ac9-8a9b-408d-ad2a-116e1c480a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:06.140525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135ac9-8a9b-408d-ad2a-116e1c480a15 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:06.141361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135ac9-8a9b-408d-ad2a-116e1c480a15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:06.141672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135ac9-8a9b-408d-ad2a-116e1c480a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:06.143071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135ac9-8a9b-408d-ad2a-116e1c480a15 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:06.153467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 38/81] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:06.155718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135ac9-8a9b-408d-ad2a-116e1c480a15 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:06.172916 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71913b0-ebaf-47b7-9e8a-27530da0fdc2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:06.218843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c559b62-9df7-4e9b-9e40-ce7c20bde482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:06.220020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c559b62-9df7-4e9b-9e40-ce7c20bde482 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:06.220470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c559b62-9df7-4e9b-9e40-ce7c20bde482 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:06.221008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c559b62-9df7-4e9b-9e40-ce7c20bde482 None admin] PATH_INFO: `/v3/users/c60d4a5f5d83459c99e38bd82d610465` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:06.222398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c559b62-9df7-4e9b-9e40-ce7c20bde482 None admin] RBAC: Authorizing `identity:get_user(user_id=c60d4a5f5d83459c99e38bd82d610465)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:06.224071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c559b62-9df7-4e9b-9e40-ce7c20bde482 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:06.225931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 44/82] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:06 2022] GET /identity/v3/users/c60d4a5f5d83459c99e38bd82d610465 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:06.226868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c559b62-9df7-4e9b-9e40-ce7c20bde482 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:06.241660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:06.279731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135ac9-8a9b-408d-ad2a-116e1c480a15 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:06.322782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088de664-ea4c-41a8-a864-80c0b05e2fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:06.324045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088de664-ea4c-41a8-a864-80c0b05e2fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:06.324403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088de664-ea4c-41a8-a864-80c0b05e2fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:06.324774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088de664-ea4c-41a8-a864-80c0b05e2fc7 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:06.326061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088de664-ea4c-41a8-a864-80c0b05e2fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:06.328198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088de664-ea4c-41a8-a864-80c0b05e2fc7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:06.330874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 39/83] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:06 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:06.332511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088de664-ea4c-41a8-a864-80c0b05e2fc7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:06.336191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:06.342699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c559b62-9df7-4e9b-9e40-ce7c20bde482 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:06.382235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988716c-e635-4391-9b22-51807a9edff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:06.383316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988716c-e635-4391-9b22-51807a9edff3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:06.383737 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988716c-e635-4391-9b22-51807a9edff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:06.384156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988716c-e635-4391-9b22-51807a9edff3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:06.398654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988716c-e635-4391-9b22-51807a9edff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4988716c-e635-4391-9b22-51807a9edff3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:06.407336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:06.408165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:06.408952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:06.411454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 45/84] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:06.412562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988716c-e635-4391-9b22-51807a9edff3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:06.457407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088de664-ea4c-41a8-a864-80c0b05e2fc7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:06.496673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:06.502689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9596d5-38d8-4c55-9b4d-7ec21cbcffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:06.503584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9596d5-38d8-4c55-9b4d-7ec21cbcffa3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:06.503863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9596d5-38d8-4c55-9b4d-7ec21cbcffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:06.504137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9596d5-38d8-4c55-9b4d-7ec21cbcffa3 None admin] PATH_INFO: `/v3/users/86fea4d6526441fa9d310035bfcc8012` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:06.505412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9596d5-38d8-4c55-9b4d-7ec21cbcffa3 None admin] RBAC: Authorizing `identity:get_user(user_id=86fea4d6526441fa9d310035bfcc8012)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:06.507471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9596d5-38d8-4c55-9b4d-7ec21cbcffa3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:06.510381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 40/85] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:06 2022] GET /identity/v3/users/86fea4d6526441fa9d310035bfcc8012 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:06.511307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9596d5-38d8-4c55-9b4d-7ec21cbcffa3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:06.534381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988716c-e635-4391-9b22-51807a9edff3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:06.577440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5ae49-c0a0-4de9-a459-b18775350678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:06.578521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5ae49-c0a0-4de9-a459-b18775350678 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:06.578811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5ae49-c0a0-4de9-a459-b18775350678 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:06.579417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5ae49-c0a0-4de9-a459-b18775350678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:06.596024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:06.629991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9596d5-38d8-4c55-9b4d-7ec21cbcffa3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:06.664839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5ae49-c0a0-4de9-a459-b18775350678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:06.666068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1696e02-1918-473a-8f16-1fcf34eb918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:06.666304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5ae49-c0a0-4de9-a459-b18775350678 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:06.667262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1696e02-1918-473a-8f16-1fcf34eb918e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:06.667861 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1696e02-1918-473a-8f16-1fcf34eb918e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:06.668431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1696e02-1918-473a-8f16-1fcf34eb918e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:06.669541 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1696e02-1918-473a-8f16-1fcf34eb918e None admin] RBAC: Adding query filter params (role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=8ae24634160b4859b608dd29275497f2, user.id=c60d4a5f5d83459c99e38bd82d610465) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:06.669955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1696e02-1918-473a-8f16-1fcf34eb918e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:06.671548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1696e02-1918-473a-8f16-1fcf34eb918e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:06.680347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 41/86] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:06 2022] GET /identity/v3/role_assignments?user.id=c60d4a5f5d83459c99e38bd82d610465&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:06.681344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1696e02-1918-473a-8f16-1fcf34eb918e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:06.757123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:06.790102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 46/87] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:06.791061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5ae49-c0a0-4de9-a459-b18775350678 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:06.870529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1696e02-1918-473a-8f16-1fcf34eb918e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:06.922279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5ae49-c0a0-4de9-a459-b18775350678 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:06.934344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3180e14-7852-4e49-85fd-82dc8168a632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:06.935035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3180e14-7852-4e49-85fd-82dc8168a632 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:06.935323 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3180e14-7852-4e49-85fd-82dc8168a632 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:06.935601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3180e14-7852-4e49-85fd-82dc8168a632 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:06.936585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3180e14-7852-4e49-85fd-82dc8168a632 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:06.936932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3180e14-7852-4e49-85fd-82dc8168a632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:06.938441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3180e14-7852-4e49-85fd-82dc8168a632 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:06.949157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:06.961026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 42/88] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:06.961716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3180e14-7852-4e49-85fd-82dc8168a632 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:06.965892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91607e0-e0bf-4a5d-a4cb-7832388b12c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:06.966577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91607e0-e0bf-4a5d-a4cb-7832388b12c9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:06.966864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91607e0-e0bf-4a5d-a4cb-7832388b12c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:06.967145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91607e0-e0bf-4a5d-a4cb-7832388b12c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:07.069619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3180e14-7852-4e49-85fd-82dc8168a632 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:07.101675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:07.109467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569252b-f172-46db-8310-54bc9cce47b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:07.110346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569252b-f172-46db-8310-54bc9cce47b0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:07.110627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569252b-f172-46db-8310-54bc9cce47b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:07.110899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569252b-f172-46db-8310-54bc9cce47b0 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:07.112112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569252b-f172-46db-8310-54bc9cce47b0 None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:07.113625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569252b-f172-46db-8310-54bc9cce47b0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:07.115763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 43/89] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:06 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:07.116537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569252b-f172-46db-8310-54bc9cce47b0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:07.189988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:07.201416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91607e0-e0bf-4a5d-a4cb-7832388b12c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:07.203012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91607e0-e0bf-4a5d-a4cb-7832388b12c9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:07.244614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569252b-f172-46db-8310-54bc9cce47b0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:07.282162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c465f-c083-4d58-b352-059d61e8fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:07.283056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c465f-c083-4d58-b352-059d61e8fa42 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:07.283543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c465f-c083-4d58-b352-059d61e8fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:07.283826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c465f-c083-4d58-b352-059d61e8fa42 None admin] PATH_INFO: `/v3/users/c60d4a5f5d83459c99e38bd82d610465` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:07.285166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c465f-c083-4d58-b352-059d61e8fa42 None admin] RBAC: Authorizing `identity:get_user(user_id=c60d4a5f5d83459c99e38bd82d610465)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:07.286684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c465f-c083-4d58-b352-059d61e8fa42 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:07.288420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 44/90] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:07 2022] GET /identity/v3/users/c60d4a5f5d83459c99e38bd82d610465 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:07.289321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c465f-c083-4d58-b352-059d61e8fa42 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:07.399140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c465f-c083-4d58-b352-059d61e8fa42 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:07.437102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:07.449052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b87253-be23-423f-ba1e-816228a0b0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:07.449052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b87253-be23-423f-ba1e-816228a0b0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:07.449052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b87253-be23-423f-ba1e-816228a0b0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:07.449052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b87253-be23-423f-ba1e-816228a0b0b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:07.449395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b87253-be23-423f-ba1e-816228a0b0b3 None admin] RBAC: Adding query filter params (role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=8ae24634160b4859b608dd29275497f2, user.id=c60d4a5f5d83459c99e38bd82d610465) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:07.449395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b87253-be23-423f-ba1e-816228a0b0b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:07.449395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b87253-be23-423f-ba1e-816228a0b0b3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:07.457968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 45/91] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:07 2022] GET /identity/v3/role_assignments?user.id=c60d4a5f5d83459c99e38bd82d610465&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:07.458921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b87253-be23-423f-ba1e-816228a0b0b3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:07.519985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 47/92] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:07.526647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91607e0-e0bf-4a5d-a4cb-7832388b12c9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:07.668122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b87253-be23-423f-ba1e-816228a0b0b3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:07.731801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:07.738478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21da0e1-89fb-4a74-9193-1bdac8066376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:07.739149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da0e1-89fb-4a74-9193-1bdac8066376 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:07.739434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da0e1-89fb-4a74-9193-1bdac8066376 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:07.739717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da0e1-89fb-4a74-9193-1bdac8066376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:07.815546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91607e0-e0bf-4a5d-a4cb-7832388b12c9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:07.838148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21da0e1-89fb-4a74-9193-1bdac8066376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:07.840304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21da0e1-89fb-4a74-9193-1bdac8066376 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:07.859150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:07.925461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29719758-4796-4d5c-b35b-ae63e0f707da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:07.926624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29719758-4796-4d5c-b35b-ae63e0f707da None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:07.927047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29719758-4796-4d5c-b35b-ae63e0f707da None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:07.927459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29719758-4796-4d5c-b35b-ae63e0f707da None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:07.928852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29719758-4796-4d5c-b35b-ae63e0f707da None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:07.930478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29719758-4796-4d5c-b35b-ae63e0f707da None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:07.932641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 48/93] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:07 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:07.938775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29719758-4796-4d5c-b35b-ae63e0f707da None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:08.024596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 46/94] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:08.030011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21da0e1-89fb-4a74-9193-1bdac8066376 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:08.206713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29719758-4796-4d5c-b35b-ae63e0f707da None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:08.238330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:08.251460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21da0e1-89fb-4a74-9193-1bdac8066376 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:08.282037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae56f2-0a0a-4409-ba0b-6fc31ef525d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:08.282737 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae56f2-0a0a-4409-ba0b-6fc31ef525d7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:08.283031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae56f2-0a0a-4409-ba0b-6fc31ef525d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:08.283313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae56f2-0a0a-4409-ba0b-6fc31ef525d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:08.284172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae56f2-0a0a-4409-ba0b-6fc31ef525d7 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=8ae24634160b4859b608dd29275497f2, user.id=86fea4d6526441fa9d310035bfcc8012) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:08.284489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae56f2-0a0a-4409-ba0b-6fc31ef525d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:08.285986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae56f2-0a0a-4409-ba0b-6fc31ef525d7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:08.297796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 49/95] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:07 2022] GET /identity/v3/role_assignments?user.id=86fea4d6526441fa9d310035bfcc8012&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 259 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:08.303533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fec498-29e3-4cf4-99d7-94d9b33d1949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:08.304759 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fec498-29e3-4cf4-99d7-94d9b33d1949 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:08.305258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fec498-29e3-4cf4-99d7-94d9b33d1949 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:08.305739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fec498-29e3-4cf4-99d7-94d9b33d1949 None admin] PATH_INFO: `/v3/projects/b13778cce4d0406190d10632d25b4f88` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:08.307371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fec498-29e3-4cf4-99d7-94d9b33d1949 None admin] RBAC: Authorizing `identity:get_project(project_id=b13778cce4d0406190d10632d25b4f88)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:08.309142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fec498-29e3-4cf4-99d7-94d9b33d1949 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:08.311289 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 47/96] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:08 2022] GET /identity/v3/projects/b13778cce4d0406190d10632d25b4f88 => generated 312 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:08.317565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae56f2-0a0a-4409-ba0b-6fc31ef525d7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:08.323257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:08.484127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae56f2-0a0a-4409-ba0b-6fc31ef525d7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:08.537707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adf54e-e99b-4287-8ec0-f73420a56b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:08.538316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adf54e-e99b-4287-8ec0-f73420a56b24 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:08.538606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adf54e-e99b-4287-8ec0-f73420a56b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:08.538889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adf54e-e99b-4287-8ec0-f73420a56b24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:08.539769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adf54e-e99b-4287-8ec0-f73420a56b24 None admin] RBAC: Adding query filter params (role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=b13778cce4d0406190d10632d25b4f88, user.id=c60d4a5f5d83459c99e38bd82d610465) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:08.540086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adf54e-e99b-4287-8ec0-f73420a56b24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:08.541606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adf54e-e99b-4287-8ec0-f73420a56b24 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:08.549503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 50/97] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:08 2022] GET /identity/v3/role_assignments?user.id=c60d4a5f5d83459c99e38bd82d610465&scope.project.id=b13778cce4d0406190d10632d25b4f88&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:08.552626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df356a4d-4ec9-460a-b035-8d48da97f3eb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:08.553221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df356a4d-4ec9-460a-b035-8d48da97f3eb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:08.553698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df356a4d-4ec9-460a-b035-8d48da97f3eb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:08.555313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 48/98] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:08.561506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14fbd1-5a76-4d51-88db-b23dbbbdf995 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:08.562135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14fbd1-5a76-4d51-88db-b23dbbbdf995 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:08.562631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14fbd1-5a76-4d51-88db-b23dbbbdf995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:08.582951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0c14fbd1-5a76-4d51-88db-b23dbbbdf995 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:08.584474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0c14fbd1-5a76-4d51-88db-b23dbbbdf995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:08.656970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:08.658219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 51/99] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:08.680461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df356a4d-4ec9-460a-b035-8d48da97f3eb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:08.801729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df356a4d-4ec9-460a-b035-8d48da97f3eb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:08.838727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda6a4a-00d4-424d-b2cf-6ef0a7dcc89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:08.839529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda6a4a-00d4-424d-b2cf-6ef0a7dcc89b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:08.839813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda6a4a-00d4-424d-b2cf-6ef0a7dcc89b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:08.840085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda6a4a-00d4-424d-b2cf-6ef0a7dcc89b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:08.841282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda6a4a-00d4-424d-b2cf-6ef0a7dcc89b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:08.842851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda6a4a-00d4-424d-b2cf-6ef0a7dcc89b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:08.844643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 49/100] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:08.850455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14fbd1-5a76-4d51-88db-b23dbbbdf995 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:08.871997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2e8ff-70fe-49e7-a0b8-eccd1931493d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:08.872334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2e8ff-70fe-49e7-a0b8-eccd1931493d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:08.872552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2e8ff-70fe-49e7-a0b8-eccd1931493d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:08.874034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 50/101] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:08.879568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06712597-3fca-4a76-b4ba-3c4151fa5588 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:08.879859 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06712597-3fca-4a76-b4ba-3c4151fa5588 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:08.880141 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06712597-3fca-4a76-b4ba-3c4151fa5588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:08.916076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-06712597-3fca-4a76-b4ba-3c4151fa5588 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:08.917313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-06712597-3fca-4a76-b4ba-3c4151fa5588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:08.966870 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:08.983262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14fbd1-5a76-4d51-88db-b23dbbbdf995 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:09.021585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b7b6e-2287-4107-a8ae-283553c800cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:09.022775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b7b6e-2287-4107-a8ae-283553c800cd None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:09.023195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b7b6e-2287-4107-a8ae-283553c800cd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:09.023691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b7b6e-2287-4107-a8ae-283553c800cd None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:09.025621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b7b6e-2287-4107-a8ae-283553c800cd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:09.027406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b7b6e-2287-4107-a8ae-283553c800cd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:09.053579 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 51/102] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 24 10:22:09.078148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 52/103] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:22:08 2022] POST /identity/v3/projects => generated 302 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:09.106840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06712597-3fca-4a76-b4ba-3c4151fa5588 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:09.240602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06712597-3fca-4a76-b4ba-3c4151fa5588 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:09.286031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae24ce-e495-4854-bc7d-647e271096ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:09.287094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae24ce-e495-4854-bc7d-647e271096ef None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:09.287512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae24ce-e495-4854-bc7d-647e271096ef None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:09.287919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae24ce-e495-4854-bc7d-647e271096ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:09.304068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae24ce-e495-4854-bc7d-647e271096ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-26ae24ce-e495-4854-bc7d-647e271096ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:09.312346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:09.313424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:09.314338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:09.316702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 52/104] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:09.326459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b7b6e-2287-4107-a8ae-283553c800cd None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:09.389936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:09.448907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b7b6e-2287-4107-a8ae-283553c800cd None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:09.487765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd1d58-cbf5-4268-814e-7f83ca29e6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:09.488449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd1d58-cbf5-4268-814e-7f83ca29e6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:09.488769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd1d58-cbf5-4268-814e-7f83ca29e6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:09.489054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd1d58-cbf5-4268-814e-7f83ca29e6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:09.489824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd1d58-cbf5-4268-814e-7f83ca29e6c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:09.490137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd1d58-cbf5-4268-814e-7f83ca29e6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:09.491551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd1d58-cbf5-4268-814e-7f83ca29e6c5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:09.500574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 53/105] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:09.505841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae24ce-e495-4854-bc7d-647e271096ef None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:09.536081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:09.593714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023cc31-c528-4c15-a359-7377e3d8010b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:09.594002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023cc31-c528-4c15-a359-7377e3d8010b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:09.594236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023cc31-c528-4c15-a359-7377e3d8010b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:09.595651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 54/106] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:09.600957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c9b1d-f7b3-4091-aa0e-41aeae4701c4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:09.601507 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c9b1d-f7b3-4091-aa0e-41aeae4701c4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:09.602627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c9b1d-f7b3-4091-aa0e-41aeae4701c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:09.625098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f50c9b1d-f7b3-4091-aa0e-41aeae4701c4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:09.628175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f50c9b1d-f7b3-4091-aa0e-41aeae4701c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:09.632338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae24ce-e495-4854-bc7d-647e271096ef None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:09.690383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c009-e854-4d0a-82ef-586598cc867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:09.691972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c009-e854-4d0a-82ef-586598cc867d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:09.692396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c009-e854-4d0a-82ef-586598cc867d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:09.692868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c009-e854-4d0a-82ef-586598cc867d None admin] PATH_INFO: `/v3/users/af3a8bff79184a40a740b7e97812f304` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:09.725509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c009-e854-4d0a-82ef-586598cc867d None admin] RBAC: Authorizing `identity:get_user(user_id=af3a8bff79184a40a740b7e97812f304)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:09.727010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c009-e854-4d0a-82ef-586598cc867d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:09.728988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 53/107] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:09 2022] GET /identity/v3/users/af3a8bff79184a40a740b7e97812f304 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:09.735395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c009-e854-4d0a-82ef-586598cc867d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:09.738370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 55/108] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:09.772971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c9b1d-f7b3-4091-aa0e-41aeae4701c4 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:09.839651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:09.840448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:09.862586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c009-e854-4d0a-82ef-586598cc867d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:09.900321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd94035-dfd0-4b2b-b5f3-b7174111cd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:09.901480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd94035-dfd0-4b2b-b5f3-b7174111cd53 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:09.901970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd94035-dfd0-4b2b-b5f3-b7174111cd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:09.902489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd94035-dfd0-4b2b-b5f3-b7174111cd53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:09.931088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c9b1d-f7b3-4091-aa0e-41aeae4701c4 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:09.980987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd94035-dfd0-4b2b-b5f3-b7174111cd53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:09.984539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd94035-dfd0-4b2b-b5f3-b7174111cd53 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:09.989586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818b4d5-352f-487b-8453-5d5fd9b25fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:09.990431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818b4d5-352f-487b-8453-5d5fd9b25fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:09.990718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818b4d5-352f-487b-8453-5d5fd9b25fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:09.991158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818b4d5-352f-487b-8453-5d5fd9b25fb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:10.005964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818b4d5-352f-487b-8453-5d5fd9b25fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-2818b4d5-352f-487b-8453-5d5fd9b25fb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:10.022356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:10.023338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:10.024230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:10.026571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 56/109] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:10.032576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818b4d5-352f-487b-8453-5d5fd9b25fb3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:10.090370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:10.112676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 54/110] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:10.114345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8c9c5-5fd6-4018-abd5-e689acbdaab4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:10.114831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8c9c5-5fd6-4018-abd5-e689acbdaab4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:10.115304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8c9c5-5fd6-4018-abd5-e689acbdaab4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:10.116867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 55/111] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:10.119975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8c9c5-5fd6-4018-abd5-e689acbdaab4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:10.141223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:10.143387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818b4d5-352f-487b-8453-5d5fd9b25fb3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:10.178081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1746cc5-0de2-4811-ab16-472dfd9766cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:10.178726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1746cc5-0de2-4811-ab16-472dfd9766cf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:10.179012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1746cc5-0de2-4811-ab16-472dfd9766cf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:10.179352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1746cc5-0de2-4811-ab16-472dfd9766cf None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:10.180112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1746cc5-0de2-4811-ab16-472dfd9766cf None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:10.180423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1746cc5-0de2-4811-ab16-472dfd9766cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:10.181967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1746cc5-0de2-4811-ab16-472dfd9766cf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:10.190667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 57/112] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:10.191745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a88a9c-7449-41b7-9cbf-22991a92423b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:10.192095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a88a9c-7449-41b7-9cbf-22991a92423b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:10.192383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a88a9c-7449-41b7-9cbf-22991a92423b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:10.213685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-38a88a9c-7449-41b7-9cbf-22991a92423b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:10.214922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-38a88a9c-7449-41b7-9cbf-22991a92423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:10.246000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8c9c5-5fd6-4018-abd5-e689acbdaab4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:10.285928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 58/113] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:10.287672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a88a9c-7449-41b7-9cbf-22991a92423b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:10.310594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee85c8-3d4c-47b3-aae6-43e170d54ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:10.311585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee85c8-3d4c-47b3-aae6-43e170d54ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:10.311925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee85c8-3d4c-47b3-aae6-43e170d54ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:10.312153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee85c8-3d4c-47b3-aae6-43e170d54ad3 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:10.314026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee85c8-3d4c-47b3-aae6-43e170d54ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:10.315485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee85c8-3d4c-47b3-aae6-43e170d54ad3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:10.320775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 56/114] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:10 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:10.326555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee85c8-3d4c-47b3-aae6-43e170d54ad3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:10.421925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:10.450545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:10.460263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a88a9c-7449-41b7-9cbf-22991a92423b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:10.497343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ea10f-9ab2-4e25-9b5d-0918d670b686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:10.498538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ea10f-9ab2-4e25-9b5d-0918d670b686 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:10.498699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ea10f-9ab2-4e25-9b5d-0918d670b686 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:10.498940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ea10f-9ab2-4e25-9b5d-0918d670b686 None admin] PATH_INFO: `/v3/users/c60d4a5f5d83459c99e38bd82d610465` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:10.500280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ea10f-9ab2-4e25-9b5d-0918d670b686 None admin] RBAC: Authorizing `identity:get_user(user_id=c60d4a5f5d83459c99e38bd82d610465)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:10.504787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ea10f-9ab2-4e25-9b5d-0918d670b686 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:10.504787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 59/115] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:10 2022] GET /identity/v3/users/c60d4a5f5d83459c99e38bd82d610465 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:10.504787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ea10f-9ab2-4e25-9b5d-0918d670b686 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:10.554764 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee85c8-3d4c-47b3-aae6-43e170d54ad3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:10.621013 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50b271-680d-4809-9ceb-1078e91518a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:10.622528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50b271-680d-4809-9ceb-1078e91518a6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:10.623013 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50b271-680d-4809-9ceb-1078e91518a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:10.623480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50b271-680d-4809-9ceb-1078e91518a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:10.643482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50b271-680d-4809-9ceb-1078e91518a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4e50b271-680d-4809-9ceb-1078e91518a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:10.660677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:10.661549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:10.662331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:10.666922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 57/116] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:10.669661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50b271-680d-4809-9ceb-1078e91518a6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:10.704966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ea10f-9ab2-4e25-9b5d-0918d670b686 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:10.744285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7816481-5d4a-4abe-b235-6803840d3466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:10.745306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7816481-5d4a-4abe-b235-6803840d3466 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:10.745834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7816481-5d4a-4abe-b235-6803840d3466 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:10.746301 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7816481-5d4a-4abe-b235-6803840d3466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:10.747323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7816481-5d4a-4abe-b235-6803840d3466 None admin] RBAC: Adding query filter params (role.id=c0d100940f624679824541cd7101a203, scope.project.id=8ae24634160b4859b608dd29275497f2, user.id=af3a8bff79184a40a740b7e97812f304) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:10.747848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7816481-5d4a-4abe-b235-6803840d3466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:10.749636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7816481-5d4a-4abe-b235-6803840d3466 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:10.755349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 60/117] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:10 2022] GET /identity/v3/role_assignments?user.id=af3a8bff79184a40a740b7e97812f304&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=c0d100940f624679824541cd7101a203 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:10.756461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7816481-5d4a-4abe-b235-6803840d3466 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:10.799077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:10.862077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7816481-5d4a-4abe-b235-6803840d3466 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:10.904109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfa107-9332-481b-886a-cb353e6a2040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:10.906103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfa107-9332-481b-886a-cb353e6a2040 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:10.909691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfa107-9332-481b-886a-cb353e6a2040 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:10.910244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfa107-9332-481b-886a-cb353e6a2040 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:10.911242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfa107-9332-481b-886a-cb353e6a2040 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:10.911768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfa107-9332-481b-886a-cb353e6a2040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:10.913422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfa107-9332-481b-886a-cb353e6a2040 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:10.930391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 61/118] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:10.934877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfa107-9332-481b-886a-cb353e6a2040 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:10.941014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50b271-680d-4809-9ceb-1078e91518a6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:10.962954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:11.025709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73babd2-6d59-4315-b5f0-d811f2c262bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:11.026984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73babd2-6d59-4315-b5f0-d811f2c262bd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:11.027443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73babd2-6d59-4315-b5f0-d811f2c262bd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:11.027925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73babd2-6d59-4315-b5f0-d811f2c262bd None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:11.029378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73babd2-6d59-4315-b5f0-d811f2c262bd None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:11.031094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73babd2-6d59-4315-b5f0-d811f2c262bd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:11.033029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 58/119] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:10 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:11.038471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73babd2-6d59-4315-b5f0-d811f2c262bd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:11.090460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfa107-9332-481b-886a-cb353e6a2040 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:11.113175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:11.134515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990be9a-6ed7-42b8-8458-a19d25d2cd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:11.135737 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990be9a-6ed7-42b8-8458-a19d25d2cd3d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:11.136226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990be9a-6ed7-42b8-8458-a19d25d2cd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:11.136697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990be9a-6ed7-42b8-8458-a19d25d2cd3d None admin] PATH_INFO: `/v3/users/c60d4a5f5d83459c99e38bd82d610465` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:11.139355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990be9a-6ed7-42b8-8458-a19d25d2cd3d None admin] RBAC: Authorizing `identity:get_user(user_id=c60d4a5f5d83459c99e38bd82d610465)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:11.141255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990be9a-6ed7-42b8-8458-a19d25d2cd3d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:11.145759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 62/120] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:10 2022] GET /identity/v3/users/c60d4a5f5d83459c99e38bd82d610465 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:11.150120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990be9a-6ed7-42b8-8458-a19d25d2cd3d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:11.251032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73babd2-6d59-4315-b5f0-d811f2c262bd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:11.287897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990be9a-6ed7-42b8-8458-a19d25d2cd3d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:11.310833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d1e8b-1643-41cf-a931-30dabcfbab9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:11.311893 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d1e8b-1643-41cf-a931-30dabcfbab9f None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:11.312184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d1e8b-1643-41cf-a931-30dabcfbab9f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:11.312464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d1e8b-1643-41cf-a931-30dabcfbab9f None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2/users/c60d4a5f5d83459c99e38bd82d610465/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:11.316899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d1e8b-1643-41cf-a931-30dabcfbab9f None admin] RBAC: Authorizing `identity:create_grant(project_id=8ae24634160b4859b608dd29275497f2, user_id=c60d4a5f5d83459c99e38bd82d610465, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:11.318377 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d1e8b-1643-41cf-a931-30dabcfbab9f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:11.324498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9693081-68a5-4fbf-a37c-609664d05edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:11.325683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9693081-68a5-4fbf-a37c-609664d05edd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:11.326168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9693081-68a5-4fbf-a37c-609664d05edd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:11.326661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9693081-68a5-4fbf-a37c-609664d05edd None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:11.328145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9693081-68a5-4fbf-a37c-609664d05edd None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:11.330271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9693081-68a5-4fbf-a37c-609664d05edd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:11.332560 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 63/121] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:11 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:11.338764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9693081-68a5-4fbf-a37c-609664d05edd None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:11.345540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 59/122] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:22:11 2022] PUT /identity/v3/projects/8ae24634160b4859b608dd29275497f2/users/c60d4a5f5d83459c99e38bd82d610465/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:22:11.368653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:11.457167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9693081-68a5-4fbf-a37c-609664d05edd None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:11.496143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b1140-c08a-40c6-8e4b-8ef7ba087bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:11.497201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b1140-c08a-40c6-8e4b-8ef7ba087bec None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:11.497644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b1140-c08a-40c6-8e4b-8ef7ba087bec None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:11.497934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b1140-c08a-40c6-8e4b-8ef7ba087bec None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2/users/c60d4a5f5d83459c99e38bd82d610465/roles/bde14daa0b464f6f9622e4d84e28e7d1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:11.500391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b1140-c08a-40c6-8e4b-8ef7ba087bec None admin] RBAC: Authorizing `identity:create_grant(project_id=8ae24634160b4859b608dd29275497f2, user_id=c60d4a5f5d83459c99e38bd82d610465, role_id=bde14daa0b464f6f9622e4d84e28e7d1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:11.502000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b1140-c08a-40c6-8e4b-8ef7ba087bec None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:11.535165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 64/123] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:22:11 2022] PUT /identity/v3/projects/8ae24634160b4859b608dd29275497f2/users/c60d4a5f5d83459c99e38bd82d610465/roles/bde14daa0b464f6f9622e4d84e28e7d1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:22:11.780067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3e55-36c6-498a-8c47-2b827667d479 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:11.780408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3e55-36c6-498a-8c47-2b827667d479 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:11.780578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3e55-36c6-498a-8c47-2b827667d479 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:11.781963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 60/124] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:11.787620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515759ac-81c2-4ee8-b462-4d49029f349e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:11.787999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515759ac-81c2-4ee8-b462-4d49029f349e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:11.788253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515759ac-81c2-4ee8-b462-4d49029f349e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:11.810843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-515759ac-81c2-4ee8-b462-4d49029f349e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:11.812185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-515759ac-81c2-4ee8-b462-4d49029f349e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:11.869411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4f10f-de1c-49f8-81bf-6ad8b8f1a6c2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:11.869735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4f10f-de1c-49f8-81bf-6ad8b8f1a6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:11.869968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4f10f-de1c-49f8-81bf-6ad8b8f1a6c2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:11.871457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 61/125] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:11.877123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8c33-e188-4b12-a4d9-52df694b73fe None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:11.877472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8c33-e188-4b12-a4d9-52df694b73fe None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:11.877754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8c33-e188-4b12-a4d9-52df694b73fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:11.884816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 65/126] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:11.909850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-178e8c33-e188-4b12-a4d9-52df694b73fe None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:11.910391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515759ac-81c2-4ee8-b462-4d49029f349e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:11.911058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-178e8c33-e188-4b12-a4d9-52df694b73fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:11.926484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:11.966408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:12.033415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515759ac-81c2-4ee8-b462-4d49029f349e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:12.034158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 62/127] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:12.057281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e8c33-e188-4b12-a4d9-52df694b73fe None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:12.071790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daba07-e6ad-4d37-ba63-e248842ae0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:12.072608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daba07-e6ad-4d37-ba63-e248842ae0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:12.073176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daba07-e6ad-4d37-ba63-e248842ae0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:12.073467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daba07-e6ad-4d37-ba63-e248842ae0c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:12.103962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daba07-e6ad-4d37-ba63-e248842ae0c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-14daba07-e6ad-4d37-ba63-e248842ae0c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:12.116647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:12.118964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:12.120886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 66/128] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:12.129755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daba07-e6ad-4d37-ba63-e248842ae0c8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:12.192685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e8c33-e188-4b12-a4d9-52df694b73fe None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:12.239936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956f688-ed5f-4931-b7e9-e28f800650e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:12.240770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956f688-ed5f-4931-b7e9-e28f800650e6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:12.241062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956f688-ed5f-4931-b7e9-e28f800650e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:12.241338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956f688-ed5f-4931-b7e9-e28f800650e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:12.262542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:12.262842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956f688-ed5f-4931-b7e9-e28f800650e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1956f688-ed5f-4931-b7e9-e28f800650e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:12.276945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:12.279064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:12.281023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 63/129] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:22:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:12.286728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956f688-ed5f-4931-b7e9-e28f800650e6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:12.311613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daba07-e6ad-4d37-ba63-e248842ae0c8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:12.332725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:12.409581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947dd09-46ba-40dc-b369-740d1fd67228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:12.410314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947dd09-46ba-40dc-b369-740d1fd67228 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:12.410599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947dd09-46ba-40dc-b369-740d1fd67228 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:12.410880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947dd09-46ba-40dc-b369-740d1fd67228 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:12.411655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947dd09-46ba-40dc-b369-740d1fd67228 None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:12.411964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947dd09-46ba-40dc-b369-740d1fd67228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:12.413451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947dd09-46ba-40dc-b369-740d1fd67228 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:12.432638 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 67/130] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:12.444053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947dd09-46ba-40dc-b369-740d1fd67228 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:12.499296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956f688-ed5f-4931-b7e9-e28f800650e6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:12.575206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af78eb-8f2e-4b06-a7b6-c6dc92ce32f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:12.575860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af78eb-8f2e-4b06-a7b6-c6dc92ce32f1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:12.576155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af78eb-8f2e-4b06-a7b6-c6dc92ce32f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:12.576438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af78eb-8f2e-4b06-a7b6-c6dc92ce32f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:12.577260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af78eb-8f2e-4b06-a7b6-c6dc92ce32f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:12.577582 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af78eb-8f2e-4b06-a7b6-c6dc92ce32f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:12.579295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af78eb-8f2e-4b06-a7b6-c6dc92ce32f1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:12.587905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947dd09-46ba-40dc-b369-740d1fd67228 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:12.601324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 64/131] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:12.602305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3df67-a9e4-4dbe-b498-d3210025b852 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:12.602604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3df67-a9e4-4dbe-b498-d3210025b852 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:12.602894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3df67-a9e4-4dbe-b498-d3210025b852 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:12.604315 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 65/132] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:12.604909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3df67-a9e4-4dbe-b498-d3210025b852 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:12.622293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b84719-4fbc-4a7a-b87e-c9871c1903d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:12.623265 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b84719-4fbc-4a7a-b87e-c9871c1903d6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:12.623712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b84719-4fbc-4a7a-b87e-c9871c1903d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:12.624142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b84719-4fbc-4a7a-b87e-c9871c1903d6 None admin] PATH_INFO: `/v3/users/c60d4a5f5d83459c99e38bd82d610465` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:12.625440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b84719-4fbc-4a7a-b87e-c9871c1903d6 None admin] RBAC: Authorizing `identity:get_user(user_id=c60d4a5f5d83459c99e38bd82d610465)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:12.626996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b84719-4fbc-4a7a-b87e-c9871c1903d6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:12.628695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 68/133] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:12 2022] GET /identity/v3/users/c60d4a5f5d83459c99e38bd82d610465 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:12.630012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808c9e-e383-4dd2-a329-a91e1d4467fd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:12.630372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808c9e-e383-4dd2-a329-a91e1d4467fd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:12.630720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808c9e-e383-4dd2-a329-a91e1d4467fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:12.653081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:12.663148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-24808c9e-e383-4dd2-a329-a91e1d4467fd None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:12.666248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-24808c9e-e383-4dd2-a329-a91e1d4467fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:12.766923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:12.787785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3df67-a9e4-4dbe-b498-d3210025b852 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:12.790434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 69/134] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:12.791359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24808c9e-e383-4dd2-a329-a91e1d4467fd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:12.859829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716f2c8-5a90-4e37-adf2-4e0446336b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:12.861387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716f2c8-5a90-4e37-adf2-4e0446336b50 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:12.861808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716f2c8-5a90-4e37-adf2-4e0446336b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:12.862217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716f2c8-5a90-4e37-adf2-4e0446336b50 None admin] PATH_INFO: `/v3/users/86fea4d6526441fa9d310035bfcc8012` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:12.863827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716f2c8-5a90-4e37-adf2-4e0446336b50 None admin] RBAC: Authorizing `identity:get_user(user_id=86fea4d6526441fa9d310035bfcc8012)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:12.865816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716f2c8-5a90-4e37-adf2-4e0446336b50 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:12.868149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 66/135] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:12 2022] GET /identity/v3/users/86fea4d6526441fa9d310035bfcc8012 => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:12.869332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716f2c8-5a90-4e37-adf2-4e0446336b50 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:12.976465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24808c9e-e383-4dd2-a329-a91e1d4467fd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:13.011873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:13.022388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c6b58-de2c-4794-b2b6-f8147405868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:13.023569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c6b58-de2c-4794-b2b6-f8147405868f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:13.024039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c6b58-de2c-4794-b2b6-f8147405868f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:13.024502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c6b58-de2c-4794-b2b6-f8147405868f None admin] PATH_INFO: `/v3/projects/b13778cce4d0406190d10632d25b4f88` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:13.025921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c6b58-de2c-4794-b2b6-f8147405868f None admin] RBAC: Authorizing `identity:get_project(project_id=b13778cce4d0406190d10632d25b4f88)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:13.027678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c6b58-de2c-4794-b2b6-f8147405868f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:13.029736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 70/136] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:12 2022] GET /identity/v3/projects/b13778cce4d0406190d10632d25b4f88 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:13.030893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c6b58-de2c-4794-b2b6-f8147405868f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:13.090725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716f2c8-5a90-4e37-adf2-4e0446336b50 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:13.140162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c6b58-de2c-4794-b2b6-f8147405868f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:13.175037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e8cbc-f10e-4417-9a51-acf20ea61ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:13.175928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e8cbc-f10e-4417-9a51-acf20ea61ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:13.176250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e8cbc-f10e-4417-9a51-acf20ea61ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:13.176527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e8cbc-f10e-4417-9a51-acf20ea61ce4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:13.178049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e8cbc-f10e-4417-9a51-acf20ea61ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:13.178853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bfe75-c012-4eb7-8317-8777150e9b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:13.179491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e8cbc-f10e-4417-9a51-acf20ea61ce4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:13.180194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bfe75-c012-4eb7-8317-8777150e9b9a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:13.180669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bfe75-c012-4eb7-8317-8777150e9b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:13.181200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bfe75-c012-4eb7-8317-8777150e9b9a None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:13.182650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bfe75-c012-4eb7-8317-8777150e9b9a None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:13.184442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bfe75-c012-4eb7-8317-8777150e9b9a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:13.186234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 67/137] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:13.186990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 71/138] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:13 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:13.187193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e8cbc-f10e-4417-9a51-acf20ea61ce4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:13.192653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bfe75-c012-4eb7-8317-8777150e9b9a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:13.240280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:13.318028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:13.356677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e8cbc-f10e-4417-9a51-acf20ea61ce4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:13.376498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bfe75-c012-4eb7-8317-8777150e9b9a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:13.422308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4591e-c892-4605-8844-76042d24caa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:13.423232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4591e-c892-4605-8844-76042d24caa3 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:13.423508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4591e-c892-4605-8844-76042d24caa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:13.423790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4591e-c892-4605-8844-76042d24caa3 None admin] PATH_INFO: `/v3/projects/b13778cce4d0406190d10632d25b4f88/users/c60d4a5f5d83459c99e38bd82d610465/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:13.426907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975247d5-8e8f-4249-b019-53ae469bd465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:13.427801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975247d5-8e8f-4249-b019-53ae469bd465 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:13.428080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975247d5-8e8f-4249-b019-53ae469bd465 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:13.428362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975247d5-8e8f-4249-b019-53ae469bd465 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2/users/86fea4d6526441fa9d310035bfcc8012/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:13.429909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4591e-c892-4605-8844-76042d24caa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b13778cce4d0406190d10632d25b4f88, user_id=c60d4a5f5d83459c99e38bd82d610465, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:13.433615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4591e-c892-4605-8844-76042d24caa3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:13.438303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975247d5-8e8f-4249-b019-53ae469bd465 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ae24634160b4859b608dd29275497f2, user_id=86fea4d6526441fa9d310035bfcc8012, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:13.440175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975247d5-8e8f-4249-b019-53ae469bd465 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:13.452974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 68/139] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:22:13 2022] PUT /identity/v3/projects/b13778cce4d0406190d10632d25b4f88/users/c60d4a5f5d83459c99e38bd82d610465/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:13.454425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4591e-c892-4605-8844-76042d24caa3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:13.475167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 72/140] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:22:13 2022] PUT /identity/v3/projects/8ae24634160b4859b608dd29275497f2/users/86fea4d6526441fa9d310035bfcc8012/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:22:13.593146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:13.618593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4591e-c892-4605-8844-76042d24caa3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:13.624671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf27a53-d72b-4431-a696-64d02f3dabbc None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:13.624997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf27a53-d72b-4431-a696-64d02f3dabbc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:13.625268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf27a53-d72b-4431-a696-64d02f3dabbc None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:13.626730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 73/141] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:13.632452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2460aa3-7934-4c7b-a227-d019ebdbe436 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:13.632726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2460aa3-7934-4c7b-a227-d019ebdbe436 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:13.633038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2460aa3-7934-4c7b-a227-d019ebdbe436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:13.659962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c2460aa3-7934-4c7b-a227-d019ebdbe436 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:13.661731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c2460aa3-7934-4c7b-a227-d019ebdbe436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:13.695479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64099f7d-db57-4b66-8156-e3676edbd3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:13.696090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64099f7d-db57-4b66-8156-e3676edbd3f5 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:13.696377 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64099f7d-db57-4b66-8156-e3676edbd3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:13.696661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64099f7d-db57-4b66-8156-e3676edbd3f5 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:13.698205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64099f7d-db57-4b66-8156-e3676edbd3f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:13.699705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64099f7d-db57-4b66-8156-e3676edbd3f5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:13.744001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 69/142] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:13 2022] POST /identity/v3/users => generated 283 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:13.779927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 74/143] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:13.804694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64099f7d-db57-4b66-8156-e3676edbd3f5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:13.921986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64099f7d-db57-4b66-8156-e3676edbd3f5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:13.961046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55f27d-a567-43e3-8fc1-9e9a6af73f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:13.961790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55f27d-a567-43e3-8fc1-9e9a6af73f05 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:13.962073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55f27d-a567-43e3-8fc1-9e9a6af73f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:13.962352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55f27d-a567-43e3-8fc1-9e9a6af73f05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:13.970740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:13.974820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55f27d-a567-43e3-8fc1-9e9a6af73f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-9b55f27d-a567-43e3-8fc1-9e9a6af73f05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:13.984324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:13.985225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:13.986419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 70/144] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:13.992718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2460aa3-7934-4c7b-a227-d019ebdbe436 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:14.076777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:14.133345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2460aa3-7934-4c7b-a227-d019ebdbe436 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:14.172816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a4245-eb7e-4308-9f6f-38b23e6600f8 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.173338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a4245-eb7e-4308-9f6f-38b23e6600f8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.173416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a4245-eb7e-4308-9f6f-38b23e6600f8 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.175605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 71/145] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:14.182140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a62cc-224f-4c74-947a-24b4c20e6ce8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.182428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a62cc-224f-4c74-947a-24b4c20e6ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.182704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a62cc-224f-4c74-947a-24b4c20e6ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.207032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0c5a2-1f6b-46dc-b636-73bcd253d1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:14.208449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0c5a2-1f6b-46dc-b636-73bcd253d1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.209012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0c5a2-1f6b-46dc-b636-73bcd253d1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.209488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0c5a2-1f6b-46dc-b636-73bcd253d1c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.210445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0c5a2-1f6b-46dc-b636-73bcd253d1c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:14.212334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0c5a2-1f6b-46dc-b636-73bcd253d1c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:14.213979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0c5a2-1f6b-46dc-b636-73bcd253d1c1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:14.220378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-566a62cc-224f-4c74-947a-24b4c20e6ce8 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:14.224725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-566a62cc-224f-4c74-947a-24b4c20e6ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:14.233238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 75/146] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:14.234949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdcee7-f832-4c6a-bee2-9f50ab0d8431 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.235435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdcee7-f832-4c6a-bee2-9f50ab0d8431 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.235936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdcee7-f832-4c6a-bee2-9f50ab0d8431 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.237696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 76/147] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:14.242697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdcee7-f832-4c6a-bee2-9f50ab0d8431 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:14.324912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 72/148] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:14.327112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89517632-1555-4e50-8fae-5e98852efb48 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.327446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89517632-1555-4e50-8fae-5e98852efb48 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.327730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89517632-1555-4e50-8fae-5e98852efb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.348931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-89517632-1555-4e50-8fae-5e98852efb48 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:14.350130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-89517632-1555-4e50-8fae-5e98852efb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:14.388642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdcee7-f832-4c6a-bee2-9f50ab0d8431 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:14.414847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:14.439926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 73/149] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:14.441239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89517632-1555-4e50-8fae-5e98852efb48 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:14.442081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4762c9-7b37-48f2-8e4c-ebae03bfbb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:14.443240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4762c9-7b37-48f2-8e4c-ebae03bfbb17 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.443704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4762c9-7b37-48f2-8e4c-ebae03bfbb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.444268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4762c9-7b37-48f2-8e4c-ebae03bfbb17 None admin] PATH_INFO: `/v3/users/af3a8bff79184a40a740b7e97812f304` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.452636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4762c9-7b37-48f2-8e4c-ebae03bfbb17 None admin] RBAC: Authorizing `identity:get_user(user_id=af3a8bff79184a40a740b7e97812f304)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:14.458236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4762c9-7b37-48f2-8e4c-ebae03bfbb17 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:14.460260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 77/150] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:14 2022] GET /identity/v3/users/af3a8bff79184a40a740b7e97812f304 => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:14.465669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4762c9-7b37-48f2-8e4c-ebae03bfbb17 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:14.563796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:14.593960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4762c9-7b37-48f2-8e4c-ebae03bfbb17 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:14.621042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89517632-1555-4e50-8fae-5e98852efb48 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:14.632786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba4cdd-fe71-4972-a71d-3a63f72c8594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:14.632786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba4cdd-fe71-4972-a71d-3a63f72c8594 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.632786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba4cdd-fe71-4972-a71d-3a63f72c8594 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.632786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba4cdd-fe71-4972-a71d-3a63f72c8594 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.633171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba4cdd-fe71-4972-a71d-3a63f72c8594 None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:14.634959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba4cdd-fe71-4972-a71d-3a63f72c8594 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:14.636845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 78/151] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:14 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:14.636845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba4cdd-fe71-4972-a71d-3a63f72c8594 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:14.677574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288766-e2e8-4035-aff1-149b0476c93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:14.678783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288766-e2e8-4035-aff1-149b0476c93a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.679259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288766-e2e8-4035-aff1-149b0476c93a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.679734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288766-e2e8-4035-aff1-149b0476c93a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.696633 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288766-e2e8-4035-aff1-149b0476c93a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:14.698126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-76288766-e2e8-4035-aff1-149b0476c93a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:14.713687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:14.714577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:14.715370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:14.718010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 74/152] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:14.719197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288766-e2e8-4035-aff1-149b0476c93a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:14.746600 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba4cdd-fe71-4972-a71d-3a63f72c8594 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:14.783997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a8532-9387-44d8-96fa-b5964949836d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:14.784844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a8532-9387-44d8-96fa-b5964949836d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.785134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a8532-9387-44d8-96fa-b5964949836d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.785542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a8532-9387-44d8-96fa-b5964949836d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.797217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a8532-9387-44d8-96fa-b5964949836d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-6f3a8532-9387-44d8-96fa-b5964949836d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:14.807821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:14.808655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 79/153] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:14.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a8532-9387-44d8-96fa-b5964949836d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:14.929124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a8532-9387-44d8-96fa-b5964949836d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:14.955637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288766-e2e8-4035-aff1-149b0476c93a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:14.958208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:14.966766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2c4ba-9a9d-41cb-9449-ab466f5458a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:14.967513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2c4ba-9a9d-41cb-9449-ab466f5458a2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:14.967827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2c4ba-9a9d-41cb-9449-ab466f5458a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:14.968179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2c4ba-9a9d-41cb-9449-ab466f5458a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:14.969894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2c4ba-9a9d-41cb-9449-ab466f5458a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:14.970216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2c4ba-9a9d-41cb-9449-ab466f5458a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:14.972219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2c4ba-9a9d-41cb-9449-ab466f5458a2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:14.982189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 80/154] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:14.983251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2c4ba-9a9d-41cb-9449-ab466f5458a2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:15.000770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:15.044025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dc82e-a7ae-4a27-8c44-1ec679394b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:15.045420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dc82e-a7ae-4a27-8c44-1ec679394b41 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:15.045948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dc82e-a7ae-4a27-8c44-1ec679394b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:15.046422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dc82e-a7ae-4a27-8c44-1ec679394b41 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2/users/af3a8bff79184a40a740b7e97812f304/roles/c0d100940f624679824541cd7101a203` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:15.053030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dc82e-a7ae-4a27-8c44-1ec679394b41 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ae24634160b4859b608dd29275497f2, user_id=af3a8bff79184a40a740b7e97812f304, role_id=c0d100940f624679824541cd7101a203)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:15.055134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dc82e-a7ae-4a27-8c44-1ec679394b41 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:15.085288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 75/155] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:22:14 2022] PUT /identity/v3/projects/8ae24634160b4859b608dd29275497f2/users/af3a8bff79184a40a740b7e97812f304/roles/c0d100940f624679824541cd7101a203 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:15.086648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dc82e-a7ae-4a27-8c44-1ec679394b41 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:15.102490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2c4ba-9a9d-41cb-9449-ab466f5458a2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:15.139830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977b489-1fe4-419a-97c3-c173b4d88681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:15.140518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977b489-1fe4-419a-97c3-c173b4d88681 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:15.140850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977b489-1fe4-419a-97c3-c173b4d88681 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:15.141135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977b489-1fe4-419a-97c3-c173b4d88681 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:15.141897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977b489-1fe4-419a-97c3-c173b4d88681 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:15.142214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977b489-1fe4-419a-97c3-c173b4d88681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:15.143674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977b489-1fe4-419a-97c3-c173b4d88681 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:15.153533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 81/156] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:15.159138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977b489-1fe4-419a-97c3-c173b4d88681 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:15.208536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:15.263280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dc82e-a7ae-4a27-8c44-1ec679394b41 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:15.275051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977b489-1fe4-419a-97c3-c173b4d88681 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:15.311060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26913e8b-edb7-4f2d-b7c9-b40f50f97a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:15.311959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26913e8b-edb7-4f2d-b7c9-b40f50f97a68 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:15.312240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26913e8b-edb7-4f2d-b7c9-b40f50f97a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:15.312515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26913e8b-edb7-4f2d-b7c9-b40f50f97a68 None admin] PATH_INFO: `/v3/users/c60d4a5f5d83459c99e38bd82d610465` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:15.313771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26913e8b-edb7-4f2d-b7c9-b40f50f97a68 None admin] RBAC: Authorizing `identity:get_user(user_id=c60d4a5f5d83459c99e38bd82d610465)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:15.315259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26913e8b-edb7-4f2d-b7c9-b40f50f97a68 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:15.316975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 82/157] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:15 2022] GET /identity/v3/users/c60d4a5f5d83459c99e38bd82d610465 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:15.322553 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26913e8b-edb7-4f2d-b7c9-b40f50f97a68 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:15.328207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7e2f5-338d-4158-bdbf-bbbe3a7fec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:15.329733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7e2f5-338d-4158-bdbf-bbbe3a7fec02 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:15.330208 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7e2f5-338d-4158-bdbf-bbbe3a7fec02 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:15.330678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7e2f5-338d-4158-bdbf-bbbe3a7fec02 None admin] PATH_INFO: `/v3/users/c60d4a5f5d83459c99e38bd82d610465` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:15.333455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7e2f5-338d-4158-bdbf-bbbe3a7fec02 None admin] RBAC: Authorizing `identity:get_user(user_id=c60d4a5f5d83459c99e38bd82d610465)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:15.335332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7e2f5-338d-4158-bdbf-bbbe3a7fec02 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:15.337453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 76/158] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:15 2022] GET /identity/v3/users/c60d4a5f5d83459c99e38bd82d610465 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:15.342914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7e2f5-338d-4158-bdbf-bbbe3a7fec02 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:15.424794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:15.429596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26913e8b-edb7-4f2d-b7c9-b40f50f97a68 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:15.445169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:15.480762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430b1cf-6bbf-4873-a55e-2319f90e89b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:15.480762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430b1cf-6bbf-4873-a55e-2319f90e89b0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:15.480762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430b1cf-6bbf-4873-a55e-2319f90e89b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:15.480762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430b1cf-6bbf-4873-a55e-2319f90e89b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:15.523112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7e2f5-338d-4158-bdbf-bbbe3a7fec02 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:15.551316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430b1cf-6bbf-4873-a55e-2319f90e89b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:15.552819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430b1cf-6bbf-4873-a55e-2319f90e89b0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:15.578227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb883a5-4b74-493b-a4c8-305a804284f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:15.579551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb883a5-4b74-493b-a4c8-305a804284f1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:15.580038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb883a5-4b74-493b-a4c8-305a804284f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:15.580514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb883a5-4b74-493b-a4c8-305a804284f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:15.654244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 83/159] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:22:15.660080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430b1cf-6bbf-4873-a55e-2319f90e89b0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:15.670125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:15.699458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb883a5-4b74-493b-a4c8-305a804284f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:15.700970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb883a5-4b74-493b-a4c8-305a804284f1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:15.778464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430b1cf-6bbf-4873-a55e-2319f90e89b0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:15.813457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0cb59-3acc-4b84-b496-94672455b543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:15.814326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0cb59-3acc-4b84-b496-94672455b543 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:15.814615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0cb59-3acc-4b84-b496-94672455b543 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:15.814892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0cb59-3acc-4b84-b496-94672455b543 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:15.816372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0cb59-3acc-4b84-b496-94672455b543 None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:15.817997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0cb59-3acc-4b84-b496-94672455b543 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:15.819819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 84/160] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:15 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:15.825134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:15.833989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0cb59-3acc-4b84-b496-94672455b543 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:15.901355 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 77/161] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:22:15.902307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb883a5-4b74-493b-a4c8-305a804284f1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:15.920061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:15.943631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0cb59-3acc-4b84-b496-94672455b543 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:15.997890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59466912-d2f5-4530-bdfd-ff6523e8eb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:15.998632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59466912-d2f5-4530-bdfd-ff6523e8eb85 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:15.999492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59466912-d2f5-4530-bdfd-ff6523e8eb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:15.999783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59466912-d2f5-4530-bdfd-ff6523e8eb85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:16.000658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59466912-d2f5-4530-bdfd-ff6523e8eb85 None admin] RBAC: Adding query filter params (role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=8ae24634160b4859b608dd29275497f2, user.id=c60d4a5f5d83459c99e38bd82d610465) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:16.001004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59466912-d2f5-4530-bdfd-ff6523e8eb85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:16.004431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59466912-d2f5-4530-bdfd-ff6523e8eb85 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:16.013460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 85/162] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:15 2022] GET /identity/v3/role_assignments?user.id=c60d4a5f5d83459c99e38bd82d610465&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:16.070307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb883a5-4b74-493b-a4c8-305a804284f1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:16.122812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8a11d-fd78-401d-8110-59c43c3957e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:16.124239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8a11d-fd78-401d-8110-59c43c3957e0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:16.124710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8a11d-fd78-401d-8110-59c43c3957e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:16.125214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8a11d-fd78-401d-8110-59c43c3957e0 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:16.127491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8a11d-fd78-401d-8110-59c43c3957e0 None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:16.129190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8a11d-fd78-401d-8110-59c43c3957e0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:16.131181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 78/163] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:15 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:16.137905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59466912-d2f5-4530-bdfd-ff6523e8eb85 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:16.274046 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59466912-d2f5-4530-bdfd-ff6523e8eb85 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:16.311312 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afd177-f72f-4b51-8d79-c420ff177bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:16.312664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afd177-f72f-4b51-8d79-c420ff177bff None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:16.312977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afd177-f72f-4b51-8d79-c420ff177bff None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:16.313262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afd177-f72f-4b51-8d79-c420ff177bff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:16.314125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afd177-f72f-4b51-8d79-c420ff177bff None admin] RBAC: Adding query filter params (role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=8ae24634160b4859b608dd29275497f2, user.id=c60d4a5f5d83459c99e38bd82d610465) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:16.314444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afd177-f72f-4b51-8d79-c420ff177bff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:16.315915 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afd177-f72f-4b51-8d79-c420ff177bff None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:16.322706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 86/164] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:16 2022] GET /identity/v3/role_assignments?user.id=c60d4a5f5d83459c99e38bd82d610465&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:16.878525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d05b47-f04a-465e-b09c-ba9eeda02241 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:16.879619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d05b47-f04a-465e-b09c-ba9eeda02241 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:16.880100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d05b47-f04a-465e-b09c-ba9eeda02241 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:16.881715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 79/165] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:16.887176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87b4a9-6de2-43c8-ae5c-36f3b6d74b49 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:16.887470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87b4a9-6de2-43c8-ae5c-36f3b6d74b49 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:16.887752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87b4a9-6de2-43c8-ae5c-36f3b6d74b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:16.894694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:16.908883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cc87b4a9-6de2-43c8-ae5c-36f3b6d74b49 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:16.910116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cc87b4a9-6de2-43c8-ae5c-36f3b6d74b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:16.984847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 87/166] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:17.025225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d05b47-f04a-465e-b09c-ba9eeda02241 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:17.057458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:17.130627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc75180-44f5-446d-956e-86d3fd086ea4 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:17.130917 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc75180-44f5-446d-956e-86d3fd086ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:17.131118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc75180-44f5-446d-956e-86d3fd086ea4 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:17.132519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 88/167] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:17.137855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4af5bd-5a62-4600-84db-eb153e09cd5d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:17.138144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4af5bd-5a62-4600-84db-eb153e09cd5d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:17.138427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4af5bd-5a62-4600-84db-eb153e09cd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:17.156575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d05b47-f04a-465e-b09c-ba9eeda02241 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:17.168691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-dd4af5bd-5a62-4600-84db-eb153e09cd5d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:17.169924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dd4af5bd-5a62-4600-84db-eb153e09cd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:17.202601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d830af-d52d-4cab-9ecd-fc3bb1a1c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:17.203780 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d830af-d52d-4cab-9ecd-fc3bb1a1c32a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:17.204623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d830af-d52d-4cab-9ecd-fc3bb1a1c32a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:17.205146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d830af-d52d-4cab-9ecd-fc3bb1a1c32a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:17.223003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d830af-d52d-4cab-9ecd-fc3bb1a1c32a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e0d830af-d52d-4cab-9ecd-fc3bb1a1c32a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:17.234941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:17.236135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:17.236958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:17.239524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 80/168] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:17.246525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d830af-d52d-4cab-9ecd-fc3bb1a1c32a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:17.359649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 89/169] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:17.407575 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4af5bd-5a62-4600-84db-eb153e09cd5d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:17.418798 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:17.464469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d830af-d52d-4cab-9ecd-fc3bb1a1c32a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:17.534221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64663368-d45c-4018-8816-cc6824e5192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:17.534901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64663368-d45c-4018-8816-cc6824e5192a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:17.535187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64663368-d45c-4018-8816-cc6824e5192a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:17.535465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64663368-d45c-4018-8816-cc6824e5192a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:17.536233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64663368-d45c-4018-8816-cc6824e5192a None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:17.536543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64663368-d45c-4018-8816-cc6824e5192a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:17.537957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64663368-d45c-4018-8816-cc6824e5192a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:17.553171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:17.565808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 81/170] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:17.573714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64663368-d45c-4018-8816-cc6824e5192a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:17.641239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4af5bd-5a62-4600-84db-eb153e09cd5d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:17.710618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fe68c-29ac-487d-b7b1-aedb0286ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:17.711455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fe68c-29ac-487d-b7b1-aedb0286ac9a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:17.711742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fe68c-29ac-487d-b7b1-aedb0286ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:17.712010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fe68c-29ac-487d-b7b1-aedb0286ac9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:17.730932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fe68c-29ac-487d-b7b1-aedb0286ac9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f37fe68c-29ac-487d-b7b1-aedb0286ac9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:17.743381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:17.744244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:17.745502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 90/171] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:22:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:17.746313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64663368-d45c-4018-8816-cc6824e5192a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:17.750571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fe68c-29ac-487d-b7b1-aedb0286ac9a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:17.780426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe19e0-c864-47fb-b7c3-f6a407db7ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:17.781341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe19e0-c864-47fb-b7c3-f6a407db7ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:17.781624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe19e0-c864-47fb-b7c3-f6a407db7ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:17.781898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe19e0-c864-47fb-b7c3-f6a407db7ec3 None admin] PATH_INFO: `/v3/users/c60d4a5f5d83459c99e38bd82d610465` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:17.783131 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe19e0-c864-47fb-b7c3-f6a407db7ec3 None admin] RBAC: Authorizing `identity:get_user(user_id=c60d4a5f5d83459c99e38bd82d610465)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:17.784598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe19e0-c864-47fb-b7c3-f6a407db7ec3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:17.786291 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 82/172] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:17 2022] GET /identity/v3/users/c60d4a5f5d83459c99e38bd82d610465 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:17.791625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe19e0-c864-47fb-b7c3-f6a407db7ec3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:17.801960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:17.895069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe19e0-c864-47fb-b7c3-f6a407db7ec3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:17.929837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68027c44-56b3-4f9a-8edd-48ffc25379e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:17.930618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68027c44-56b3-4f9a-8edd-48ffc25379e0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:17.930970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68027c44-56b3-4f9a-8edd-48ffc25379e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:17.931309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68027c44-56b3-4f9a-8edd-48ffc25379e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:17.936670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fe68c-29ac-487d-b7b1-aedb0286ac9a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:17.976134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:17.998876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e57916-57aa-4969-8e15-c6de934a35de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:17.999620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e57916-57aa-4969-8e15-c6de934a35de None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:17.999906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e57916-57aa-4969-8e15-c6de934a35de None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.000183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e57916-57aa-4969-8e15-c6de934a35de None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.000970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e57916-57aa-4969-8e15-c6de934a35de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:18.001280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e57916-57aa-4969-8e15-c6de934a35de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:18.001555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68027c44-56b3-4f9a-8edd-48ffc25379e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:18.002682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e57916-57aa-4969-8e15-c6de934a35de None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:18.003136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68027c44-56b3-4f9a-8edd-48ffc25379e0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:18.014946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 91/173] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:18.020526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e57916-57aa-4969-8e15-c6de934a35de None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:18.027667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:18.107733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 83/174] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:22:18.113408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68027c44-56b3-4f9a-8edd-48ffc25379e0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:18.212124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68027c44-56b3-4f9a-8edd-48ffc25379e0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:18.221126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:18.242976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e57916-57aa-4969-8e15-c6de934a35de None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:18.248784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006498a0-467d-4e2c-b87c-0283b171443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:18.249724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006498a0-467d-4e2c-b87c-0283b171443c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.250069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006498a0-467d-4e2c-b87c-0283b171443c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.250411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006498a0-467d-4e2c-b87c-0283b171443c None admin] PATH_INFO: `/v3/projects/b13778cce4d0406190d10632d25b4f88` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.254782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006498a0-467d-4e2c-b87c-0283b171443c None admin] RBAC: Authorizing `identity:get_project(project_id=b13778cce4d0406190d10632d25b4f88)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:18.254782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006498a0-467d-4e2c-b87c-0283b171443c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:18.254999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 84/175] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:18 2022] GET /identity/v3/projects/b13778cce4d0406190d10632d25b4f88 => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:18.260953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006498a0-467d-4e2c-b87c-0283b171443c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:18.314353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e757a0-d1bc-4efe-b052-56a9bb9275d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:18.315508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e757a0-d1bc-4efe-b052-56a9bb9275d4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.315948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e757a0-d1bc-4efe-b052-56a9bb9275d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.316346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e757a0-d1bc-4efe-b052-56a9bb9275d4 None admin] PATH_INFO: `/v3/users/86fea4d6526441fa9d310035bfcc8012` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.321448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e757a0-d1bc-4efe-b052-56a9bb9275d4 None admin] RBAC: Authorizing `identity:get_user(user_id=86fea4d6526441fa9d310035bfcc8012)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:18.323098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e757a0-d1bc-4efe-b052-56a9bb9275d4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:18.324942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 92/176] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:18 2022] GET /identity/v3/users/86fea4d6526441fa9d310035bfcc8012 => generated 247 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:18.330830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e757a0-d1bc-4efe-b052-56a9bb9275d4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:18.365920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006498a0-467d-4e2c-b87c-0283b171443c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:18.404361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f9d3b-de38-4391-9571-be61c558f2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:18.405410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f9d3b-de38-4391-9571-be61c558f2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.405847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f9d3b-de38-4391-9571-be61c558f2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.406268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f9d3b-de38-4391-9571-be61c558f2b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.407296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f9d3b-de38-4391-9571-be61c558f2b3 None admin] RBAC: Adding query filter params (role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=b13778cce4d0406190d10632d25b4f88, user.id=c60d4a5f5d83459c99e38bd82d610465) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:18.407754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f9d3b-de38-4391-9571-be61c558f2b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:18.409210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:18.409980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f9d3b-de38-4391-9571-be61c558f2b3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:18.417899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 85/177] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:18 2022] GET /identity/v3/role_assignments?user.id=c60d4a5f5d83459c99e38bd82d610465&scope.project.id=b13778cce4d0406190d10632d25b4f88&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:18.419403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6ca34-d9c1-4a71-bb56-2c6785640699 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.419878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6ca34-d9c1-4a71-bb56-2c6785640699 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.420346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6ca34-d9c1-4a71-bb56-2c6785640699 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.421921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 86/178] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:18.423358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8c2b2-00cf-44dd-a8cb-53fb012e338c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.423857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8c2b2-00cf-44dd-a8cb-53fb012e338c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.424314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8c2b2-00cf-44dd-a8cb-53fb012e338c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.425812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 87/179] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:18.431546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c815cc-89d5-4c22-a4fa-160c5eebc6d2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.432209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c815cc-89d5-4c22-a4fa-160c5eebc6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.432700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c815cc-89d5-4c22-a4fa-160c5eebc6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.461850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-71c815cc-89d5-4c22-a4fa-160c5eebc6d2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:18.463157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-71c815cc-89d5-4c22-a4fa-160c5eebc6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:18.498959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e757a0-d1bc-4efe-b052-56a9bb9275d4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:18.530023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:18.547058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649e311-12fa-42af-ac36-13c38410417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:18.548070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649e311-12fa-42af-ac36-13c38410417c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.548498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649e311-12fa-42af-ac36-13c38410417c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.548970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649e311-12fa-42af-ac36-13c38410417c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.593003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 88/180] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:18.595022 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0ce0a-5d54-4f00-a4f1-d8baadab2a29 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.595316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0ce0a-5d54-4f00-a4f1-d8baadab2a29 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.595594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0ce0a-5d54-4f00-a4f1-d8baadab2a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.615438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c1e0ce0a-5d54-4f00-a4f1-d8baadab2a29 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:18.616660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c1e0ce0a-5d54-4f00-a4f1-d8baadab2a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:18.661965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649e311-12fa-42af-ac36-13c38410417c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:18.661965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649e311-12fa-42af-ac36-13c38410417c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:18.706042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 89/181] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:18.706926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0ce0a-5d54-4f00-a4f1-d8baadab2a29 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:18.727000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:18.776966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 93/182] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:22:18.776966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbadd7-e078-493c-8598-607b08dccd22 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.777211 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbadd7-e078-493c-8598-607b08dccd22 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.777341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbadd7-e078-493c-8598-607b08dccd22 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.782820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 94/183] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:18.783813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbadd7-e078-493c-8598-607b08dccd22 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:18.820249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:18.825005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0ce0a-5d54-4f00-a4f1-d8baadab2a29 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:18.862445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad119f7-f914-4af0-abb1-0839595ba801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:18.863267 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad119f7-f914-4af0-abb1-0839595ba801 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.863549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad119f7-f914-4af0-abb1-0839595ba801 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.863822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad119f7-f914-4af0-abb1-0839595ba801 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.875438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad119f7-f914-4af0-abb1-0839595ba801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2ad119f7-f914-4af0-abb1-0839595ba801 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:18.883725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:18.884635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:18.885441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 90/184] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:18.886075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad119f7-f914-4af0-abb1-0839595ba801 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:18.897448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbadd7-e078-493c-8598-607b08dccd22 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:18.938924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8bd00-c856-49bc-b79c-092228790678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:18.939719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8bd00-c856-49bc-b79c-092228790678 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.940001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8bd00-c856-49bc-b79c-092228790678 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.940279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8bd00-c856-49bc-b79c-092228790678 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.951680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8bd00-c856-49bc-b79c-092228790678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-75b8bd00-c856-49bc-b79c-092228790678 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:18.959652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:18.960514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 95/185] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:22:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:18.965437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a856e-3ed8-4854-bed4-6d7ac8d0c1e6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.966159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a856e-3ed8-4854-bed4-6d7ac8d0c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.966159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a856e-3ed8-4854-bed4-6d7ac8d0c1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.979930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1e5a856e-3ed8-4854-bed4-6d7ac8d0c1e6 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:18.980814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1e5a856e-3ed8-4854-bed4-6d7ac8d0c1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:18.991506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:18.999595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad119f7-f914-4af0-abb1-0839595ba801 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.035826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17275e14-968f-4cd0-a634-e7497e48c50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.036616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17275e14-968f-4cd0-a634-e7497e48c50f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.036923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17275e14-968f-4cd0-a634-e7497e48c50f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.037201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17275e14-968f-4cd0-a634-e7497e48c50f None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.038386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17275e14-968f-4cd0-a634-e7497e48c50f None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:19.039860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17275e14-968f-4cd0-a634-e7497e48c50f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:19.046378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 91/186] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:18 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:19.046378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17275e14-968f-4cd0-a634-e7497e48c50f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.057247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 96/187] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:19.066609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a856e-3ed8-4854-bed4-6d7ac8d0c1e6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.069674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:19.140821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17275e14-968f-4cd0-a634-e7497e48c50f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.165097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a856e-3ed8-4854-bed4-6d7ac8d0c1e6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.173625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f5097-b056-4384-8f23-ab50efd39696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.174300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f5097-b056-4384-8f23-ab50efd39696 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.174573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f5097-b056-4384-8f23-ab50efd39696 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.174844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f5097-b056-4384-8f23-ab50efd39696 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.175608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f5097-b056-4384-8f23-ab50efd39696 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:19.175916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f5097-b056-4384-8f23-ab50efd39696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:19.177324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f5097-b056-4384-8f23-ab50efd39696 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:19.185543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 92/188] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:19.190410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f38e4-c433-42f7-91af-fa22d32eb4f4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.190700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f38e4-c433-42f7-91af-fa22d32eb4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.190987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f38e4-c433-42f7-91af-fa22d32eb4f4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.192288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 93/189] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:19.196274 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f38e4-c433-42f7-91af-fa22d32eb4f4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.199848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846475a0-e9a3-4b30-90ab-ff84a9597265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.200548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846475a0-e9a3-4b30-90ab-ff84a9597265 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.200861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846475a0-e9a3-4b30-90ab-ff84a9597265 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.201136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846475a0-e9a3-4b30-90ab-ff84a9597265 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.201979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846475a0-e9a3-4b30-90ab-ff84a9597265 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:19.202280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846475a0-e9a3-4b30-90ab-ff84a9597265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:19.203666 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846475a0-e9a3-4b30-90ab-ff84a9597265 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:19.212246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 97/190] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:19.213112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846475a0-e9a3-4b30-90ab-ff84a9597265 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.221284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:19.284536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:19.297562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f38e4-c433-42f7-91af-fa22d32eb4f4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.309139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846475a0-e9a3-4b30-90ab-ff84a9597265 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.330434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782e033-1f8c-4998-bfca-9a279ccefaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.331088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782e033-1f8c-4998-bfca-9a279ccefaef None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.331368 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782e033-1f8c-4998-bfca-9a279ccefaef None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.331656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782e033-1f8c-4998-bfca-9a279ccefaef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.332489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782e033-1f8c-4998-bfca-9a279ccefaef None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=8ae24634160b4859b608dd29275497f2, user.id=86fea4d6526441fa9d310035bfcc8012) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:19.332832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782e033-1f8c-4998-bfca-9a279ccefaef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:19.334271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782e033-1f8c-4998-bfca-9a279ccefaef None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:19.354448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 94/191] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:19 2022] GET /identity/v3/role_assignments?user.id=86fea4d6526441fa9d310035bfcc8012&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:19.354448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782e033-1f8c-4998-bfca-9a279ccefaef None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.354731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe2f8b-f61c-4591-baf3-bac7acd0cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.354731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe2f8b-f61c-4591-baf3-bac7acd0cae7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.354731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe2f8b-f61c-4591-baf3-bac7acd0cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.354731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe2f8b-f61c-4591-baf3-bac7acd0cae7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.355057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe2f8b-f61c-4591-baf3-bac7acd0cae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:19.355057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe2f8b-f61c-4591-baf3-bac7acd0cae7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:19.355057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 98/192] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:19.355057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c8d2-1d52-466b-9cbb-c16a20b2950b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.355057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c8d2-1d52-466b-9cbb-c16a20b2950b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.355057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c8d2-1d52-466b-9cbb-c16a20b2950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.368056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-baa5c8d2-1d52-466b-9cbb-c16a20b2950b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:19.369316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-baa5c8d2-1d52-466b-9cbb-c16a20b2950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:19.449033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 99/193] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:19.461686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5c8d2-1d52-466b-9cbb-c16a20b2950b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.464064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782e033-1f8c-4998-bfca-9a279ccefaef None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.466658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:19.507589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a88fd-fdd1-4715-9805-44859e293422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.508477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a88fd-fdd1-4715-9805-44859e293422 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.508783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a88fd-fdd1-4715-9805-44859e293422 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.509081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a88fd-fdd1-4715-9805-44859e293422 None admin] PATH_INFO: `/v3/users/660cf3927e2e440abdc14d58a574457e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.527569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a88fd-fdd1-4715-9805-44859e293422 None admin] RBAC: Authorizing `identity:get_user(user_id=660cf3927e2e440abdc14d58a574457e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:19.528396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a88fd-fdd1-4715-9805-44859e293422 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:19.534716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 95/194] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:19 2022] GET /identity/v3/users/660cf3927e2e440abdc14d58a574457e => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:19.534716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a88fd-fdd1-4715-9805-44859e293422 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.549829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:19.587194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5c8d2-1d52-466b-9cbb-c16a20b2950b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.619637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf8d3b-8b11-4e48-b439-44d92bfc2856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.620520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf8d3b-8b11-4e48-b439-44d92bfc2856 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.620834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf8d3b-8b11-4e48-b439-44d92bfc2856 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.621115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf8d3b-8b11-4e48-b439-44d92bfc2856 None admin] PATH_INFO: `/v3/users/660cf3927e2e440abdc14d58a574457e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.622297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf8d3b-8b11-4e48-b439-44d92bfc2856 None admin] RBAC: Authorizing `identity:get_user(user_id=660cf3927e2e440abdc14d58a574457e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:19.623802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf8d3b-8b11-4e48-b439-44d92bfc2856 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:19.625571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 100/195] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:19 2022] GET /identity/v3/users/660cf3927e2e440abdc14d58a574457e => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:19.626333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf8d3b-8b11-4e48-b439-44d92bfc2856 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.642396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a88fd-fdd1-4715-9805-44859e293422 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.678316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7000f-ac27-4593-b3d1-5ae7a754fdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.679390 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7000f-ac27-4593-b3d1-5ae7a754fdc4 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.679674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7000f-ac27-4593-b3d1-5ae7a754fdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.679941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7000f-ac27-4593-b3d1-5ae7a754fdc4 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.680647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7000f-ac27-4593-b3d1-5ae7a754fdc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:19.682124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7000f-ac27-4593-b3d1-5ae7a754fdc4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:19.698490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:19.702548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 96/196] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:19 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:19.703437 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7000f-ac27-4593-b3d1-5ae7a754fdc4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.733572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf8d3b-8b11-4e48-b439-44d92bfc2856 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.775379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9247e-e68d-4713-9a3c-7c822819ff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.776201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9247e-e68d-4713-9a3c-7c822819ff82 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.776483 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9247e-e68d-4713-9a3c-7c822819ff82 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.776779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9247e-e68d-4713-9a3c-7c822819ff82 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.787238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:19.789476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9247e-e68d-4713-9a3c-7c822819ff82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-8ca9247e-e68d-4713-9a3c-7c822819ff82 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:19.797789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:19.798757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:19.799741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 101/197] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:19.805057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9247e-e68d-4713-9a3c-7c822819ff82 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:19.828982 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7000f-ac27-4593-b3d1-5ae7a754fdc4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.867455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cea50a-5661-4bfc-abfa-31491cae4a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.868272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cea50a-5661-4bfc-abfa-31491cae4a12 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.869396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cea50a-5661-4bfc-abfa-31491cae4a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.869396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cea50a-5661-4bfc-abfa-31491cae4a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:19.917714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9247e-e68d-4713-9a3c-7c822819ff82 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:19.945249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cea50a-5661-4bfc-abfa-31491cae4a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:19.947016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cea50a-5661-4bfc-abfa-31491cae4a12 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:19.965614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:19.973492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df77d3d-2ae3-486e-9eaa-0c00a87e4c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:19.974305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df77d3d-2ae3-486e-9eaa-0c00a87e4c49 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:19.974536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df77d3d-2ae3-486e-9eaa-0c00a87e4c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:19.974931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df77d3d-2ae3-486e-9eaa-0c00a87e4c49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.028930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:20.065861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df77d3d-2ae3-486e-9eaa-0c00a87e4c49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:20.067803 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df77d3d-2ae3-486e-9eaa-0c00a87e4c49 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:20.071514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 97/198] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:20.072439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cea50a-5661-4bfc-abfa-31491cae4a12 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:20.176201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cea50a-5661-4bfc-abfa-31491cae4a12 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:20.197001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 102/199] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:20.197873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df77d3d-2ae3-486e-9eaa-0c00a87e4c49 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:20.214457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747c18b-be48-447c-9a27-b3cb019eee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:20.215249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747c18b-be48-447c-9a27-b3cb019eee9e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:20.215529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747c18b-be48-447c-9a27-b3cb019eee9e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:20.215802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747c18b-be48-447c-9a27-b3cb019eee9e None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.216712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747c18b-be48-447c-9a27-b3cb019eee9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:20.217455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747c18b-be48-447c-9a27-b3cb019eee9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:20.219099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747c18b-be48-447c-9a27-b3cb019eee9e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:20.229600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 98/200] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:20.230591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747c18b-be48-447c-9a27-b3cb019eee9e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:20.237575 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:20.267354 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:20.321011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df77d3d-2ae3-486e-9eaa-0c00a87e4c49 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:20.346563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747c18b-be48-447c-9a27-b3cb019eee9e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:20.366782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527daea-61d1-4ca7-b9ed-48b7f9d4dee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:20.367724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527daea-61d1-4ca7-b9ed-48b7f9d4dee3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:20.368006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527daea-61d1-4ca7-b9ed-48b7f9d4dee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:20.368282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527daea-61d1-4ca7-b9ed-48b7f9d4dee3 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.370774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527daea-61d1-4ca7-b9ed-48b7f9d4dee3 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:20.372252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527daea-61d1-4ca7-b9ed-48b7f9d4dee3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:20.374100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 103/201] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:20 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:20.374868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527daea-61d1-4ca7-b9ed-48b7f9d4dee3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:20.390029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95479f31-ca7a-4265-984f-d9c6be9b4b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:20.391962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95479f31-ca7a-4265-984f-d9c6be9b4b2d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:20.392436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95479f31-ca7a-4265-984f-d9c6be9b4b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:20.393183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95479f31-ca7a-4265-984f-d9c6be9b4b2d None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.394972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95479f31-ca7a-4265-984f-d9c6be9b4b2d None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:20.396613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95479f31-ca7a-4265-984f-d9c6be9b4b2d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:20.399011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 99/202] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:20 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:20.400082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95479f31-ca7a-4265-984f-d9c6be9b4b2d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:20.477330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:20.480886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527daea-61d1-4ca7-b9ed-48b7f9d4dee3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:20.500283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:20.507725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95479f31-ca7a-4265-984f-d9c6be9b4b2d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:20.518727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729682-03db-4b32-a140-2795d76d6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:20.519620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729682-03db-4b32-a140-2795d76d6fba None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:20.520048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729682-03db-4b32-a140-2795d76d6fba None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:20.520167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729682-03db-4b32-a140-2795d76d6fba None admin] PATH_INFO: `/v3/users/af3a8bff79184a40a740b7e97812f304` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.521393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac729682-03db-4b32-a140-2795d76d6fba None admin] RBAC: Authorizing `identity:get_user(user_id=af3a8bff79184a40a740b7e97812f304)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:20.522885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac729682-03db-4b32-a140-2795d76d6fba None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:20.529227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 104/203] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:20 2022] GET /identity/v3/users/af3a8bff79184a40a740b7e97812f304 => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:20.530181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729682-03db-4b32-a140-2795d76d6fba None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:20.545768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bbfa1-62e0-4e15-8dcf-7cad4e3085b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:20.546794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bbfa1-62e0-4e15-8dcf-7cad4e3085b6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:20.547271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bbfa1-62e0-4e15-8dcf-7cad4e3085b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:20.547725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bbfa1-62e0-4e15-8dcf-7cad4e3085b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.548757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66bbfa1-62e0-4e15-8dcf-7cad4e3085b6 None admin] RBAC: Adding query filter params (role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=46bc7b47b92148ffb91da999460d824f, user.id=660cf3927e2e440abdc14d58a574457e) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:20.549258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66bbfa1-62e0-4e15-8dcf-7cad4e3085b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:20.550913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66bbfa1-62e0-4e15-8dcf-7cad4e3085b6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:20.556902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 100/204] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:20 2022] GET /identity/v3/role_assignments?user.id=660cf3927e2e440abdc14d58a574457e&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:20.558063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bbfa1-62e0-4e15-8dcf-7cad4e3085b6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:20.640751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729682-03db-4b32-a140-2795d76d6fba None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:20.668509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bbfa1-62e0-4e15-8dcf-7cad4e3085b6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:20.678220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4431f-83a9-4b8e-994d-605b5a66dc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:20.679188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4431f-83a9-4b8e-994d-605b5a66dc88 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:20.679188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4431f-83a9-4b8e-994d-605b5a66dc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:20.679426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4431f-83a9-4b8e-994d-605b5a66dc88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.680303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4431f-83a9-4b8e-994d-605b5a66dc88 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=46bc7b47b92148ffb91da999460d824f, user.id=660cf3927e2e440abdc14d58a574457e) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:20.680621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4431f-83a9-4b8e-994d-605b5a66dc88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:20.682259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4431f-83a9-4b8e-994d-605b5a66dc88 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:20.688809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 105/205] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:20 2022] GET /identity/v3/role_assignments?user.id=660cf3927e2e440abdc14d58a574457e&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:20.715404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68983d-c3fc-4460-a582-476efba94a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:20.716420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68983d-c3fc-4460-a582-476efba94a27 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:20.716863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68983d-c3fc-4460-a582-476efba94a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:20.717318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68983d-c3fc-4460-a582-476efba94a27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.748781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:20.814427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68983d-c3fc-4460-a582-476efba94a27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:20.815980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68983d-c3fc-4460-a582-476efba94a27 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:20.924979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 101/206] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:20.930554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4431f-83a9-4b8e-994d-605b5a66dc88 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:20.980124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:21.061039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4431f-83a9-4b8e-994d-605b5a66dc88 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:21.098881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2b421-ad1c-4128-b190-986b6e78e70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:21.099770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2b421-ad1c-4128-b190-986b6e78e70c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:21.100053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2b421-ad1c-4128-b190-986b6e78e70c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:21.100326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2b421-ad1c-4128-b190-986b6e78e70c None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:21.101593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2b421-ad1c-4128-b190-986b6e78e70c None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:21.103080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2b421-ad1c-4128-b190-986b6e78e70c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:21.105160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 106/207] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:20 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:21.111385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68983d-c3fc-4460-a582-476efba94a27 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:21.204881 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:21.255244 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68983d-c3fc-4460-a582-476efba94a27 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:21.288930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fb8ea-f4ea-44d2-a317-d3b773bfd507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:21.289934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fb8ea-f4ea-44d2-a317-d3b773bfd507 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:21.290404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fb8ea-f4ea-44d2-a317-d3b773bfd507 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:21.290862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fb8ea-f4ea-44d2-a317-d3b773bfd507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:21.291899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fb8ea-f4ea-44d2-a317-d3b773bfd507 None admin] RBAC: Adding query filter params (role.id=c0d100940f624679824541cd7101a203, scope.project.id=8ae24634160b4859b608dd29275497f2, user.id=af3a8bff79184a40a740b7e97812f304) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:21.292394 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fb8ea-f4ea-44d2-a317-d3b773bfd507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:21.294048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fb8ea-f4ea-44d2-a317-d3b773bfd507 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:21.300321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 102/208] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:21 2022] GET /identity/v3/role_assignments?user.id=af3a8bff79184a40a740b7e97812f304&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=c0d100940f624679824541cd7101a203 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:22.065322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374888ec-0b56-4791-9ed5-c39e5f143e13 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.065839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374888ec-0b56-4791-9ed5-c39e5f143e13 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.065944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374888ec-0b56-4791-9ed5-c39e5f143e13 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.067710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 107/209] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:22.073506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ae637-c933-4371-9b2b-9e168307be26 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.073812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ae637-c933-4371-9b2b-9e168307be26 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.074060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ae637-c933-4371-9b2b-9e168307be26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.091562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-198ae637-c933-4371-9b2b-9e168307be26 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:22.092825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-198ae637-c933-4371-9b2b-9e168307be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:22.131899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bf38a-d1e9-443f-a54d-924b27e29477 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.132198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bf38a-d1e9-443f-a54d-924b27e29477 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.132465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bf38a-d1e9-443f-a54d-924b27e29477 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.134113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 108/210] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:22.139655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e97320-ec0a-4080-a33e-d6a40bae16f3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.140144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e97320-ec0a-4080-a33e-d6a40bae16f3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.140459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e97320-ec0a-4080-a33e-d6a40bae16f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.157922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-08e97320-ec0a-4080-a33e-d6a40bae16f3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:22.159383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-08e97320-ec0a-4080-a33e-d6a40bae16f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:22.165470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 103/211] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:22.188595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ae637-c933-4371-9b2b-9e168307be26 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:22.230207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 109/212] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:22.255375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:22.255792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e97320-ec0a-4080-a33e-d6a40bae16f3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:22.305179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:22.310708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ae637-c933-4371-9b2b-9e168307be26 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:22.354130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae59f5-fffc-4fde-8463-92a303bcb3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:22.355549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae59f5-fffc-4fde-8463-92a303bcb3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.355969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae59f5-fffc-4fde-8463-92a303bcb3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.356374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae59f5-fffc-4fde-8463-92a303bcb3a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.370308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae59f5-fffc-4fde-8463-92a303bcb3a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:22.374783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e97320-ec0a-4080-a33e-d6a40bae16f3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2aae59f5-fffc-4fde-8463-92a303bcb3a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:22.378523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:22.379571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:22.380523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:22.382298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 104/213] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:22.387963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae59f5-fffc-4fde-8463-92a303bcb3a7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:22.416148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16ced-38f6-495d-8a34-d9cb725a8547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:22.417362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16ced-38f6-495d-8a34-d9cb725a8547 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.417809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16ced-38f6-495d-8a34-d9cb725a8547 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.418306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16ced-38f6-495d-8a34-d9cb725a8547 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.419667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be16ced-38f6-495d-8a34-d9cb725a8547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:22.421593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be16ced-38f6-495d-8a34-d9cb725a8547 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:22.423754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 110/214] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:22.429400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16ced-38f6-495d-8a34-d9cb725a8547 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:22.524168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae59f5-fffc-4fde-8463-92a303bcb3a7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:22.542641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:22.552191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:22.556613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16ced-38f6-495d-8a34-d9cb725a8547 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:22.570785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c919bd-7c57-4cb4-b5ae-98e740df816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:22.571684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c919bd-7c57-4cb4-b5ae-98e740df816c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.572136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c919bd-7c57-4cb4-b5ae-98e740df816c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.572661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c919bd-7c57-4cb4-b5ae-98e740df816c None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.573955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c919bd-7c57-4cb4-b5ae-98e740df816c None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:22.574519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c919bd-7c57-4cb4-b5ae-98e740df816c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:22.576309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c919bd-7c57-4cb4-b5ae-98e740df816c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:22.587459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 105/215] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:22.593538 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c919bd-7c57-4cb4-b5ae-98e740df816c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:22.600631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eede76-e9a9-4351-b951-e81b673dd787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:22.601388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eede76-e9a9-4351-b951-e81b673dd787 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.601674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eede76-e9a9-4351-b951-e81b673dd787 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.601955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eede76-e9a9-4351-b951-e81b673dd787 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.602681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eede76-e9a9-4351-b951-e81b673dd787 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:22.604303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eede76-e9a9-4351-b951-e81b673dd787 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:22.628705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 111/216] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:22 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:22:22.719210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c919bd-7c57-4cb4-b5ae-98e740df816c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:22.757693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df6150-fd9c-4138-baa2-18fceabedc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:22.758967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df6150-fd9c-4138-baa2-18fceabedc5e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.759498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df6150-fd9c-4138-baa2-18fceabedc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.759978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df6150-fd9c-4138-baa2-18fceabedc5e None admin] PATH_INFO: `/v3/users/6058a77019bd47ae89dcc6997ae81f08` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.777181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df6150-fd9c-4138-baa2-18fceabedc5e None admin] RBAC: Authorizing `identity:get_user(user_id=6058a77019bd47ae89dcc6997ae81f08)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:22.779061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df6150-fd9c-4138-baa2-18fceabedc5e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:22.781422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 106/217] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:22 2022] GET /identity/v3/users/6058a77019bd47ae89dcc6997ae81f08 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:22.790198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eede76-e9a9-4351-b951-e81b673dd787 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:22.900973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eede76-e9a9-4351-b951-e81b673dd787 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:22.907565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf619f38-255f-40cb-84bb-4a4e155aa2db None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.907842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf619f38-255f-40cb-84bb-4a4e155aa2db None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.908125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf619f38-255f-40cb-84bb-4a4e155aa2db None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.909555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 107/218] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:22.914831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ff11f-4093-4d5b-ae1b-f2b13e5d195c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.915118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ff11f-4093-4d5b-ae1b-f2b13e5d195c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.915398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ff11f-4093-4d5b-ae1b-f2b13e5d195c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.936328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5c0ff11f-4093-4d5b-ae1b-f2b13e5d195c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:22.938028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5c0ff11f-4093-4d5b-ae1b-f2b13e5d195c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:22.944055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ad3fe-606d-4f8d-ac27-9ca3bd5e3922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:22.945024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ad3fe-606d-4f8d-ac27-9ca3bd5e3922 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:22.945596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ad3fe-606d-4f8d-ac27-9ca3bd5e3922 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:22.946565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ad3fe-606d-4f8d-ac27-9ca3bd5e3922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:22.968679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:22.975621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:23.028173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 108/219] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:23.029502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8586-3d9c-4f40-911e-0e9cc4bc4e29 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:23.029856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8586-3d9c-4f40-911e-0e9cc4bc4e29 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:23.030163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8586-3d9c-4f40-911e-0e9cc4bc4e29 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:23.031605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 109/220] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:23.037809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5cb80-7bd7-454d-8fb8-44bfb39ff2f6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:23.037809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5cb80-7bd7-454d-8fb8-44bfb39ff2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:23.038094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5cb80-7bd7-454d-8fb8-44bfb39ff2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:23.040246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ad3fe-606d-4f8d-ac27-9ca3bd5e3922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:23.041902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ad3fe-606d-4f8d-ac27-9ca3bd5e3922 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:23.057966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-43b5cb80-7bd7-454d-8fb8-44bfb39ff2f6 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:23.059395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-43b5cb80-7bd7-454d-8fb8-44bfb39ff2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:23.151225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 110/221] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:23.151817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5cb80-7bd7-454d-8fb8-44bfb39ff2f6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:23.155081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 112/222] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:23.165791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ad3fe-606d-4f8d-ac27-9ca3bd5e3922 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:23.240337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:23.272557 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:23.282305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5cb80-7bd7-454d-8fb8-44bfb39ff2f6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:23.297125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ad3fe-606d-4f8d-ac27-9ca3bd5e3922 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:23.318958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770e49e-062a-4f2f-810f-182f7f1273c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:23.319767 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770e49e-062a-4f2f-810f-182f7f1273c6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:23.320049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770e49e-062a-4f2f-810f-182f7f1273c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:23.320321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770e49e-062a-4f2f-810f-182f7f1273c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:23.331551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770e49e-062a-4f2f-810f-182f7f1273c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b770e49e-062a-4f2f-810f-182f7f1273c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:23.339268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:23.340150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 111/223] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:22:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:23.341360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770e49e-062a-4f2f-810f-182f7f1273c6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:23.346652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffb358-878f-45ab-b868-dbc1ad2f407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:23.347871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffb358-878f-45ab-b868-dbc1ad2f407b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:23.348340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffb358-878f-45ab-b868-dbc1ad2f407b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:23.348854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffb358-878f-45ab-b868-dbc1ad2f407b None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:23.350270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffb358-878f-45ab-b868-dbc1ad2f407b None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:23.352645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffb358-878f-45ab-b868-dbc1ad2f407b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:23.355596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 113/224] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:23 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:23.357045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffb358-878f-45ab-b868-dbc1ad2f407b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:23.455369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770e49e-062a-4f2f-810f-182f7f1273c6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:23.473835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffb358-878f-45ab-b868-dbc1ad2f407b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:23.491350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8091588-62d4-4101-a5a4-cdb813aa3707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:23.492163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8091588-62d4-4101-a5a4-cdb813aa3707 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:23.492446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8091588-62d4-4101-a5a4-cdb813aa3707 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:23.492718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8091588-62d4-4101-a5a4-cdb813aa3707 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:23.499927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:23.505476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8091588-62d4-4101-a5a4-cdb813aa3707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:23.513555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10db46a-af9b-41ca-928e-bd9bc8fb3ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:23.514598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10db46a-af9b-41ca-928e-bd9bc8fb3ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b8091588-62d4-4101-a5a4-cdb813aa3707 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:23.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:23.515831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:23.516595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:23.517910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10db46a-af9b-41ca-928e-bd9bc8fb3ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:23.518407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10db46a-af9b-41ca-928e-bd9bc8fb3ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:23.519369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10db46a-af9b-41ca-928e-bd9bc8fb3ef4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:23.519872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10db46a-af9b-41ca-928e-bd9bc8fb3ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:23.521250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 112/225] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:23.521827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10db46a-af9b-41ca-928e-bd9bc8fb3ef4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:23.522638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8091588-62d4-4101-a5a4-cdb813aa3707 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:23.532060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 114/226] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:23.533370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10db46a-af9b-41ca-928e-bd9bc8fb3ef4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:23.542178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:23.645143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8091588-62d4-4101-a5a4-cdb813aa3707 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:23.658903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10db46a-af9b-41ca-928e-bd9bc8fb3ef4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:23.684078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22861006-b458-436b-a7cc-253fd1a53df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:23.685245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22861006-b458-436b-a7cc-253fd1a53df3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:23.685787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22861006-b458-436b-a7cc-253fd1a53df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:23.686392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22861006-b458-436b-a7cc-253fd1a53df3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:23.687475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22861006-b458-436b-a7cc-253fd1a53df3 None admin] RBAC: Adding query filter params (role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=46bc7b47b92148ffb91da999460d824f, user.id=6058a77019bd47ae89dcc6997ae81f08) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:23.687974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22861006-b458-436b-a7cc-253fd1a53df3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:23.689817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22861006-b458-436b-a7cc-253fd1a53df3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:23.696447 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 113/227] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:23 2022] GET /identity/v3/role_assignments?user.id=6058a77019bd47ae89dcc6997ae81f08&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:23.697534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22861006-b458-436b-a7cc-253fd1a53df3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:23.703549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa4380-e762-44e3-9803-70454504c250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:23.704567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa4380-e762-44e3-9803-70454504c250 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:23.704880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa4380-e762-44e3-9803-70454504c250 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:23.705159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa4380-e762-44e3-9803-70454504c250 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:23.705991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa4380-e762-44e3-9803-70454504c250 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:23.706243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa4380-e762-44e3-9803-70454504c250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:23.707653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa4380-e762-44e3-9803-70454504c250 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:23.727447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 115/228] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:23.733960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa4380-e762-44e3-9803-70454504c250 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:23.831151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:23.864093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:23.907937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22861006-b458-436b-a7cc-253fd1a53df3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:23.934587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa4380-e762-44e3-9803-70454504c250 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:23.978066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e688bc-0ea0-480b-a4f7-3153bd4ba642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:23.978878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e688bc-0ea0-480b-a4f7-3153bd4ba642 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:23.979160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e688bc-0ea0-480b-a4f7-3153bd4ba642 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:23.979433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e688bc-0ea0-480b-a4f7-3153bd4ba642 None admin] PATH_INFO: `/v3/users/660cf3927e2e440abdc14d58a574457e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:23.980824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e688bc-0ea0-480b-a4f7-3153bd4ba642 None admin] RBAC: Authorizing `identity:get_user(user_id=660cf3927e2e440abdc14d58a574457e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:23.982294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e688bc-0ea0-480b-a4f7-3153bd4ba642 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:23.984192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 114/229] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:23 2022] GET /identity/v3/users/660cf3927e2e440abdc14d58a574457e => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:23.990060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e688bc-0ea0-480b-a4f7-3153bd4ba642 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:24.011875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee85270-4da0-4a73-b7f9-386993212766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:24.012781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85270-4da0-4a73-b7f9-386993212766 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:24.013068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85270-4da0-4a73-b7f9-386993212766 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:24.013344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85270-4da0-4a73-b7f9-386993212766 None admin] PATH_INFO: `/v3/users/660cf3927e2e440abdc14d58a574457e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:24.015209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee85270-4da0-4a73-b7f9-386993212766 None admin] RBAC: Authorizing `identity:get_user(user_id=660cf3927e2e440abdc14d58a574457e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:24.016607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee85270-4da0-4a73-b7f9-386993212766 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:24.018313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 116/230] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:23 2022] GET /identity/v3/users/660cf3927e2e440abdc14d58a574457e => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:24.034596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee85270-4da0-4a73-b7f9-386993212766 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:24.141837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e688bc-0ea0-480b-a4f7-3153bd4ba642 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:24.183882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558786c-d377-4e3e-b988-22c15db0b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:24.185554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558786c-d377-4e3e-b988-22c15db0b5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:24.185834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558786c-d377-4e3e-b988-22c15db0b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:24.186347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558786c-d377-4e3e-b988-22c15db0b5b8 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:24.189707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558786c-d377-4e3e-b988-22c15db0b5b8 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:24.191170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558786c-d377-4e3e-b988-22c15db0b5b8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:24.192906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 115/231] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:23 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:24.199230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558786c-d377-4e3e-b988-22c15db0b5b8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:24.216656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:24.268632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee85270-4da0-4a73-b7f9-386993212766 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:24.302632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:24.313328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558786c-d377-4e3e-b988-22c15db0b5b8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:24.329021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb88cd-6577-4ecb-854a-d9db347001ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:24.330207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb88cd-6577-4ecb-854a-d9db347001ad None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:24.330486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb88cd-6577-4ecb-854a-d9db347001ad None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:24.330762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb88cd-6577-4ecb-854a-d9db347001ad None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:24.332143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb88cd-6577-4ecb-854a-d9db347001ad None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:24.333733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb88cd-6577-4ecb-854a-d9db347001ad None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:24.335440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 117/232] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:24 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:24.341048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb88cd-6577-4ecb-854a-d9db347001ad None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:24.359550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851412e-fc5c-4f39-b492-3d35ea06a24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:24.360467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851412e-fc5c-4f39-b492-3d35ea06a24c None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:24.360760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851412e-fc5c-4f39-b492-3d35ea06a24c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:24.361044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851412e-fc5c-4f39-b492-3d35ea06a24c None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f/users/660cf3927e2e440abdc14d58a574457e/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:24.367704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851412e-fc5c-4f39-b492-3d35ea06a24c None admin] RBAC: Authorizing `identity:create_grant(project_id=46bc7b47b92148ffb91da999460d824f, user_id=660cf3927e2e440abdc14d58a574457e, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:24.369637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851412e-fc5c-4f39-b492-3d35ea06a24c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:24.403078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 116/233] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:22:24 2022] PUT /identity/v3/projects/46bc7b47b92148ffb91da999460d824f/users/660cf3927e2e440abdc14d58a574457e/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:22:24.459803 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb88cd-6577-4ecb-854a-d9db347001ad None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:24.501669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266b228-1e42-4899-bf1a-d82f3f58b705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:24.502578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266b228-1e42-4899-bf1a-d82f3f58b705 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:24.502848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266b228-1e42-4899-bf1a-d82f3f58b705 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:24.503124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266b228-1e42-4899-bf1a-d82f3f58b705 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f/users/660cf3927e2e440abdc14d58a574457e/roles/bde14daa0b464f6f9622e4d84e28e7d1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:24.505428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266b228-1e42-4899-bf1a-d82f3f58b705 None admin] RBAC: Authorizing `identity:create_grant(project_id=46bc7b47b92148ffb91da999460d824f, user_id=660cf3927e2e440abdc14d58a574457e, role_id=bde14daa0b464f6f9622e4d84e28e7d1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:24.506894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266b228-1e42-4899-bf1a-d82f3f58b705 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:24.539208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 118/234] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:22:24 2022] PUT /identity/v3/projects/46bc7b47b92148ffb91da999460d824f/users/660cf3927e2e440abdc14d58a574457e/roles/bde14daa0b464f6f9622e4d84e28e7d1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:22:24.990699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dade2c-0a4a-4f27-a53a-e6c98ae5596b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:24.991076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dade2c-0a4a-4f27-a53a-e6c98ae5596b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:24.991283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dade2c-0a4a-4f27-a53a-e6c98ae5596b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:24.992928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 117/235] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:24.998401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a14757-9951-44aa-a3b8-5e7a17d7040e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:24.999034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a14757-9951-44aa-a3b8-5e7a17d7040e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:24.999320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a14757-9951-44aa-a3b8-5e7a17d7040e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:25.013422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce8727-3f22-4d94-9183-ac787243495b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:25.013704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce8727-3f22-4d94-9183-ac787243495b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:25.013981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce8727-3f22-4d94-9183-ac787243495b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:25.015744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 118/236] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:25.022890 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159a7b9-44f1-4572-9776-ec740c92b436 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:25.023175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159a7b9-44f1-4572-9776-ec740c92b436 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:25.023455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159a7b9-44f1-4572-9776-ec740c92b436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:25.038893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c6a14757-9951-44aa-a3b8-5e7a17d7040e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:25.040108 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c6a14757-9951-44aa-a3b8-5e7a17d7040e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:25.048341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a159a7b9-44f1-4572-9776-ec740c92b436 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:25.049540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a159a7b9-44f1-4572-9776-ec740c92b436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:25.119719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:25.150616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 119/237] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:25.174522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159a7b9-44f1-4572-9776-ec740c92b436 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:25.178705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:25.188337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 119/238] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:25.213063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a14757-9951-44aa-a3b8-5e7a17d7040e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:25.326033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159a7b9-44f1-4572-9776-ec740c92b436 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:25.364365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a990c1e-8900-4acc-9a32-deecc82bdd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:25.366609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a990c1e-8900-4acc-9a32-deecc82bdd06 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:25.366889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a990c1e-8900-4acc-9a32-deecc82bdd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:25.367160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a990c1e-8900-4acc-9a32-deecc82bdd06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:25.369841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a990c1e-8900-4acc-9a32-deecc82bdd06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:25.372606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a990c1e-8900-4acc-9a32-deecc82bdd06 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:25.382463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 120/239] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:25.383301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a990c1e-8900-4acc-9a32-deecc82bdd06 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:25.386646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a14757-9951-44aa-a3b8-5e7a17d7040e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:25.432750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05905c3d-0f9b-4103-a917-7a25a7e5c6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:25.433623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05905c3d-0f9b-4103-a917-7a25a7e5c6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:25.433909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05905c3d-0f9b-4103-a917-7a25a7e5c6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:25.434186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05905c3d-0f9b-4103-a917-7a25a7e5c6b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:25.460993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05905c3d-0f9b-4103-a917-7a25a7e5c6b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-05905c3d-0f9b-4103-a917-7a25a7e5c6b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:25.477250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:25.478598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:25.479906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 120/240] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:25.484456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05905c3d-0f9b-4103-a917-7a25a7e5c6b9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:25.537846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:25.551174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a990c1e-8900-4acc-9a32-deecc82bdd06 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:25.578024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:25.593361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b179ddd-dd97-4616-85d8-3eab7ea828c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:25.594311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b179ddd-dd97-4616-85d8-3eab7ea828c0 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:25.594797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b179ddd-dd97-4616-85d8-3eab7ea828c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:25.595245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b179ddd-dd97-4616-85d8-3eab7ea828c0 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:25.596198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b179ddd-dd97-4616-85d8-3eab7ea828c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:25.597960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b179ddd-dd97-4616-85d8-3eab7ea828c0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:25.619756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 121/241] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:25 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:25.681688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05905c3d-0f9b-4103-a917-7a25a7e5c6b9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:25.720522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197f729-5b24-4c0c-8d08-4dbffe9a781f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:25.721595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f729-5b24-4c0c-8d08-4dbffe9a781f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:25.722069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f729-5b24-4c0c-8d08-4dbffe9a781f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:25.722537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f729-5b24-4c0c-8d08-4dbffe9a781f None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:25.723569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197f729-5b24-4c0c-8d08-4dbffe9a781f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:25.724052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197f729-5b24-4c0c-8d08-4dbffe9a781f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:25.725715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197f729-5b24-4c0c-8d08-4dbffe9a781f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:25.735261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 121/242] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:25.741076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b179ddd-dd97-4616-85d8-3eab7ea828c0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:25.874872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b179ddd-dd97-4616-85d8-3eab7ea828c0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:25.902885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:25.912191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908093be-b72c-42c1-a8a6-7f2d5d63431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:25.913410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908093be-b72c-42c1-a8a6-7f2d5d63431c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:25.913874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908093be-b72c-42c1-a8a6-7f2d5d63431c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:25.914325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908093be-b72c-42c1-a8a6-7f2d5d63431c None admin] PATH_INFO: `/v3/users/5faaed96d4fa40a7af73c8137799c781` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:25.928884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908093be-b72c-42c1-a8a6-7f2d5d63431c None admin] RBAC: Authorizing `identity:get_user(user_id=5faaed96d4fa40a7af73c8137799c781)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:25.930637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908093be-b72c-42c1-a8a6-7f2d5d63431c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:25.932622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 122/243] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:25 2022] GET /identity/v3/users/5faaed96d4fa40a7af73c8137799c781 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:25.938593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197f729-5b24-4c0c-8d08-4dbffe9a781f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:26.038374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:26.046400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197f729-5b24-4c0c-8d08-4dbffe9a781f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:26.083143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99091782-baed-4427-8313-d47c67a09940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:26.084125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99091782-baed-4427-8313-d47c67a09940 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:26.084581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99091782-baed-4427-8313-d47c67a09940 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:26.085084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99091782-baed-4427-8313-d47c67a09940 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:26.159585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99091782-baed-4427-8313-d47c67a09940 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:26.161427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99091782-baed-4427-8313-d47c67a09940 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:26.233982 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e917705-46cb-45e1-b344-4ca41ec11926 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:26.234856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e917705-46cb-45e1-b344-4ca41ec11926 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:26.235448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e917705-46cb-45e1-b344-4ca41ec11926 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:26.238248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 123/244] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:26.250671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed294aa5-ae8c-4f33-81a9-b4135e98d363 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:26.251356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed294aa5-ae8c-4f33-81a9-b4135e98d363 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:26.251847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed294aa5-ae8c-4f33-81a9-b4135e98d363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:26.273072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:26.278757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ed294aa5-ae8c-4f33-81a9-b4135e98d363 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:26.280213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ed294aa5-ae8c-4f33-81a9-b4135e98d363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:26.286621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 122/245] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:26.292119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99091782-baed-4427-8313-d47c67a09940 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:26.374133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 124/246] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:26.399972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed294aa5-ae8c-4f33-81a9-b4135e98d363 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:26.424013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99091782-baed-4427-8313-d47c67a09940 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:26.476596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e43129-84ab-4518-99cb-30e920867d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:26.477601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e43129-84ab-4518-99cb-30e920867d3c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:26.477859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e43129-84ab-4518-99cb-30e920867d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:26.478139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e43129-84ab-4518-99cb-30e920867d3c None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:26.479925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e43129-84ab-4518-99cb-30e920867d3c None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:26.481471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e43129-84ab-4518-99cb-30e920867d3c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:26.483347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 123/247] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:26 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:26.493990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e43129-84ab-4518-99cb-30e920867d3c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:26.561665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed294aa5-ae8c-4f33-81a9-b4135e98d363 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:26.594398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:26.601680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:26.631913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ec27a-ba53-44a1-82f0-fd253616ed79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:26.632721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ec27a-ba53-44a1-82f0-fd253616ed79 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:26.633062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ec27a-ba53-44a1-82f0-fd253616ed79 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:26.633332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ec27a-ba53-44a1-82f0-fd253616ed79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:26.653450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ec27a-ba53-44a1-82f0-fd253616ed79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-082ec27a-ba53-44a1-82f0-fd253616ed79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:26.664068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:26.665006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:26.665840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:26.666439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 125/248] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:26.675077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e43129-84ab-4518-99cb-30e920867d3c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:26.679738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ec27a-ba53-44a1-82f0-fd253616ed79 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:26.742253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7073185-496f-4d75-84e2-e7ac91ae541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:26.742919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7073185-496f-4d75-84e2-e7ac91ae541e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:26.743202 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7073185-496f-4d75-84e2-e7ac91ae541e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:26.743475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7073185-496f-4d75-84e2-e7ac91ae541e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:26.744333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7073185-496f-4d75-84e2-e7ac91ae541e None admin] RBAC: Adding query filter params (role.id=c0d100940f624679824541cd7101a203, scope.project.id=46bc7b47b92148ffb91da999460d824f, user.id=5faaed96d4fa40a7af73c8137799c781) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:26.744646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7073185-496f-4d75-84e2-e7ac91ae541e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:26.746142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7073185-496f-4d75-84e2-e7ac91ae541e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:26.759602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 124/249] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:26 2022] GET /identity/v3/role_assignments?user.id=5faaed96d4fa40a7af73c8137799c781&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=c0d100940f624679824541cd7101a203 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:26.842630 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ec27a-ba53-44a1-82f0-fd253616ed79 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:26.855518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccd31d-25d5-4e0d-a74d-0fa4194e2827 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:26.855824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccd31d-25d5-4e0d-a74d-0fa4194e2827 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:26.856089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccd31d-25d5-4e0d-a74d-0fa4194e2827 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:26.858045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 125/250] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:26.863584 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cf3d0-6c54-4a80-8aa8-746c45eabf5a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:26.863851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cf3d0-6c54-4a80-8aa8-746c45eabf5a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:26.864110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cf3d0-6c54-4a80-8aa8-746c45eabf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:26.882695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4b5cf3d0-6c54-4a80-8aa8-746c45eabf5a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:26.883896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4b5cf3d0-6c54-4a80-8aa8-746c45eabf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:26.907025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ef1d7-5141-46f3-a12c-dadf0d870194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:26.907727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ef1d7-5141-46f3-a12c-dadf0d870194 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:26.908011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ef1d7-5141-46f3-a12c-dadf0d870194 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:26.908286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ef1d7-5141-46f3-a12c-dadf0d870194 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:26.909141 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ef1d7-5141-46f3-a12c-dadf0d870194 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:26.909495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ef1d7-5141-46f3-a12c-dadf0d870194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:26.910930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ef1d7-5141-46f3-a12c-dadf0d870194 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:26.927287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 126/251] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:26.932722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ef1d7-5141-46f3-a12c-dadf0d870194 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:27.015296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:27.018125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 126/252] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:27.053969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cf3d0-6c54-4a80-8aa8-746c45eabf5a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:27.134192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ef1d7-5141-46f3-a12c-dadf0d870194 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:27.171087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:27.197958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870a1ea-e2fb-422c-a6e4-f18f07820803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:27.199203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870a1ea-e2fb-422c-a6e4-f18f07820803 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:27.199674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870a1ea-e2fb-422c-a6e4-f18f07820803 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:27.200116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870a1ea-e2fb-422c-a6e4-f18f07820803 None admin] PATH_INFO: `/v3/users/6058a77019bd47ae89dcc6997ae81f08` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:27.203556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870a1ea-e2fb-422c-a6e4-f18f07820803 None admin] RBAC: Authorizing `identity:get_user(user_id=6058a77019bd47ae89dcc6997ae81f08)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:27.205569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870a1ea-e2fb-422c-a6e4-f18f07820803 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:27.207438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 127/253] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:26 2022] GET /identity/v3/users/6058a77019bd47ae89dcc6997ae81f08 => generated 297 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:27.214158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870a1ea-e2fb-422c-a6e4-f18f07820803 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:27.254566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cf3d0-6c54-4a80-8aa8-746c45eabf5a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:27.342363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6ed23-19c9-48b6-8b13-aa90783fb77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:27.343479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6ed23-19c9-48b6-8b13-aa90783fb77d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:27.343931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6ed23-19c9-48b6-8b13-aa90783fb77d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:27.344400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6ed23-19c9-48b6-8b13-aa90783fb77d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:27.353511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870a1ea-e2fb-422c-a6e4-f18f07820803 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:27.361495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:27.371736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6ed23-19c9-48b6-8b13-aa90783fb77d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7cb6ed23-19c9-48b6-8b13-aa90783fb77d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:27.383752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:27.384667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:27.385469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 127/254] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:22:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:27.396167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6ed23-19c9-48b6-8b13-aa90783fb77d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:27.405535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c0612-5f3d-4e64-bae3-83a482cafe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:27.407515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0612-5f3d-4e64-bae3-83a482cafe68 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:27.408034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0612-5f3d-4e64-bae3-83a482cafe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:27.408513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0612-5f3d-4e64-bae3-83a482cafe68 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:27.410000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c0612-5f3d-4e64-bae3-83a482cafe68 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:27.411668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c0612-5f3d-4e64-bae3-83a482cafe68 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:27.413928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 128/255] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:27 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:27.415335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f94479-e3b7-428b-a2ca-36bcf1978a7e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:27.415811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f94479-e3b7-428b-a2ca-36bcf1978a7e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:27.416278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f94479-e3b7-428b-a2ca-36bcf1978a7e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:27.417727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 129/256] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:27.423791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f94479-e3b7-428b-a2ca-36bcf1978a7e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:27.545142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f94479-e3b7-428b-a2ca-36bcf1978a7e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:27.595183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bd80c-2782-4b50-882f-2f8165190961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:27.596439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bd80c-2782-4b50-882f-2f8165190961 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:27.597019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bd80c-2782-4b50-882f-2f8165190961 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:27.597525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bd80c-2782-4b50-882f-2f8165190961 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f/users/6058a77019bd47ae89dcc6997ae81f08/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:27.599703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bd80c-2782-4b50-882f-2f8165190961 None admin] RBAC: Authorizing `identity:create_grant(project_id=46bc7b47b92148ffb91da999460d824f, user_id=6058a77019bd47ae89dcc6997ae81f08, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:27.601549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bd80c-2782-4b50-882f-2f8165190961 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:27.624770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6ed23-19c9-48b6-8b13-aa90783fb77d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:27.626157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 130/257] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:22:27 2022] PUT /identity/v3/projects/46bc7b47b92148ffb91da999460d824f/users/6058a77019bd47ae89dcc6997ae81f08/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:27.627664 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a499b-0af2-4d2e-8b47-119ed0ff7113 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:27.628149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a499b-0af2-4d2e-8b47-119ed0ff7113 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:27.628710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a499b-0af2-4d2e-8b47-119ed0ff7113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:27.643781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:27.668028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-004a499b-0af2-4d2e-8b47-119ed0ff7113 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:27.668028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-004a499b-0af2-4d2e-8b47-119ed0ff7113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:27.680949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80df32-021d-4c3b-a167-d47711140023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:27.682051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80df32-021d-4c3b-a167-d47711140023 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:27.682532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80df32-021d-4c3b-a167-d47711140023 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:27.682991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80df32-021d-4c3b-a167-d47711140023 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:27.683992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80df32-021d-4c3b-a167-d47711140023 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:27.684509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80df32-021d-4c3b-a167-d47711140023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:27.686423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80df32-021d-4c3b-a167-d47711140023 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:27.705670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 128/258] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:27.710880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80df32-021d-4c3b-a167-d47711140023 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:27.716135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:27.799926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 131/259] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:27.836927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80df32-021d-4c3b-a167-d47711140023 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:27.841760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a499b-0af2-4d2e-8b47-119ed0ff7113 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:27.884351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc002a-97ac-45bf-9a46-111ff98cf727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:27.885636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc002a-97ac-45bf-9a46-111ff98cf727 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:27.886138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc002a-97ac-45bf-9a46-111ff98cf727 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:27.886598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc002a-97ac-45bf-9a46-111ff98cf727 None admin] PATH_INFO: `/v3/users/660cf3927e2e440abdc14d58a574457e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:27.893746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc002a-97ac-45bf-9a46-111ff98cf727 None admin] RBAC: Authorizing `identity:get_user(user_id=660cf3927e2e440abdc14d58a574457e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:27.895608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc002a-97ac-45bf-9a46-111ff98cf727 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:27.899594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 129/260] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:27 2022] GET /identity/v3/users/660cf3927e2e440abdc14d58a574457e => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:27.906326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc002a-97ac-45bf-9a46-111ff98cf727 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:27.962094 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:28.021632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc002a-97ac-45bf-9a46-111ff98cf727 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:28.059294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747e872-698e-4069-8f4e-648a7a2fb171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:28.059972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747e872-698e-4069-8f4e-648a7a2fb171 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:28.060278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747e872-698e-4069-8f4e-648a7a2fb171 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:28.060544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747e872-698e-4069-8f4e-648a7a2fb171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:28.066229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a499b-0af2-4d2e-8b47-119ed0ff7113 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:28.138803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152642b-4aa9-4f7c-b698-604bf375d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:28.140213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152642b-4aa9-4f7c-b698-604bf375d31f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:28.140621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152642b-4aa9-4f7c-b698-604bf375d31f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:28.140931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152642b-4aa9-4f7c-b698-604bf375d31f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:28.159574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152642b-4aa9-4f7c-b698-604bf375d31f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4152642b-4aa9-4f7c-b698-604bf375d31f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:28.175165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:28.176055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:28.177261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 132/261] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:28.184865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152642b-4aa9-4f7c-b698-604bf375d31f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:28.205703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747e872-698e-4069-8f4e-648a7a2fb171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:28.207533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747e872-698e-4069-8f4e-648a7a2fb171 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:28.225144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:28.260418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:28.318487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 130/262] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:28.325717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747e872-698e-4069-8f4e-648a7a2fb171 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:28.364161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152642b-4aa9-4f7c-b698-604bf375d31f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:28.421022 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdbf16-c9a1-49f6-b12a-2df5cdc3ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:28.421795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdbf16-c9a1-49f6-b12a-2df5cdc3ed48 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:28.422048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdbf16-c9a1-49f6-b12a-2df5cdc3ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:28.422327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdbf16-c9a1-49f6-b12a-2df5cdc3ed48 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:28.423103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdbf16-c9a1-49f6-b12a-2df5cdc3ed48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:28.423416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdbf16-c9a1-49f6-b12a-2df5cdc3ed48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:28.428970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdbf16-c9a1-49f6-b12a-2df5cdc3ed48 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:28.435832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747e872-698e-4069-8f4e-648a7a2fb171 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:28.438002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 133/263] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:28.443429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdbf16-c9a1-49f6-b12a-2df5cdc3ed48 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:28.493457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c3762-64f3-49d4-a568-5614b2b25ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:28.494703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c3762-64f3-49d4-a568-5614b2b25ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:28.495173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c3762-64f3-49d4-a568-5614b2b25ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:28.495653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c3762-64f3-49d4-a568-5614b2b25ed1 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:28.497117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c3762-64f3-49d4-a568-5614b2b25ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:28.498960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c3762-64f3-49d4-a568-5614b2b25ed1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:28.501436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 131/264] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:28 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:28.506680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c3762-64f3-49d4-a568-5614b2b25ed1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:28.531236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:28.572839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:28.582699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdbf16-c9a1-49f6-b12a-2df5cdc3ed48 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:28.646912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87051dcd-23c3-48b9-881a-d3cd38a255ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:28.647752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87051dcd-23c3-48b9-881a-d3cd38a255ad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:28.648043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87051dcd-23c3-48b9-881a-d3cd38a255ad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:28.648389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87051dcd-23c3-48b9-881a-d3cd38a255ad None admin] PATH_INFO: `/v3/users/660cf3927e2e440abdc14d58a574457e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:28.653497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87051dcd-23c3-48b9-881a-d3cd38a255ad None admin] RBAC: Authorizing `identity:get_user(user_id=660cf3927e2e440abdc14d58a574457e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:28.655415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87051dcd-23c3-48b9-881a-d3cd38a255ad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:28.658619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 134/265] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:28 2022] GET /identity/v3/users/660cf3927e2e440abdc14d58a574457e => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:28.667495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87051dcd-23c3-48b9-881a-d3cd38a255ad None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:28.694856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c3762-64f3-49d4-a568-5614b2b25ed1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:28.756406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e3b1e-beba-4190-810c-aa0da4d4d8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:28.757475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e3b1e-beba-4190-810c-aa0da4d4d8de None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:28.757970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e3b1e-beba-4190-810c-aa0da4d4d8de None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:28.758431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e3b1e-beba-4190-810c-aa0da4d4d8de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:28.759458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e3b1e-beba-4190-810c-aa0da4d4d8de None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=46bc7b47b92148ffb91da999460d824f, user.id=660cf3927e2e440abdc14d58a574457e) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:28.759957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e3b1e-beba-4190-810c-aa0da4d4d8de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:28.761629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e3b1e-beba-4190-810c-aa0da4d4d8de None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:28.775034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 132/266] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:28 2022] GET /identity/v3/role_assignments?user.id=660cf3927e2e440abdc14d58a574457e&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:28.903262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87051dcd-23c3-48b9-881a-d3cd38a255ad None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:28.958841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fdbe2-68c7-4ecc-8747-7106c1c14961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:28.959521 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fdbe2-68c7-4ecc-8747-7106c1c14961 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:28.959804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fdbe2-68c7-4ecc-8747-7106c1c14961 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:28.960078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fdbe2-68c7-4ecc-8747-7106c1c14961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:28.998157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:29.037765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db1eb2-a2e1-4365-824a-26e372128976 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.038386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db1eb2-a2e1-4365-824a-26e372128976 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.038854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db1eb2-a2e1-4365-824a-26e372128976 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.040455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 133/267] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:29.052120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fdbe2-68c7-4ecc-8747-7106c1c14961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:29.053645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fdbe2-68c7-4ecc-8747-7106c1c14961 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:29.053858 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0865f-5f73-4469-9835-d30efc9acd7f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.054326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0865f-5f73-4469-9835-d30efc9acd7f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.054816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0865f-5f73-4469-9835-d30efc9acd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.083167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a3a0865f-5f73-4469-9835-d30efc9acd7f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:29.084362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a3a0865f-5f73-4469-9835-d30efc9acd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:29.218165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 134/268] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:29.234481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 135/269] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:22:29.239699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0865f-5f73-4469-9835-d30efc9acd7f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:29.251591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:29.320081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fdbe2-68c7-4ecc-8747-7106c1c14961 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:29.370924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0865f-5f73-4469-9835-d30efc9acd7f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:29.406310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:29.423122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf0982-1a04-45a1-aa58-8966ef805743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:29.424060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf0982-1a04-45a1-aa58-8966ef805743 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.424346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf0982-1a04-45a1-aa58-8966ef805743 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.424625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf0982-1a04-45a1-aa58-8966ef805743 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.425838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf0982-1a04-45a1-aa58-8966ef805743 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:29.427309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf0982-1a04-45a1-aa58-8966ef805743 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:29.429338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 135/270] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:29 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:29.437151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178408c7-e7db-47ae-85ba-c2dc0b9400b8 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.437440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178408c7-e7db-47ae-85ba-c2dc0b9400b8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.437718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178408c7-e7db-47ae-85ba-c2dc0b9400b8 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.440205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 136/271] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:29.444591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafc8f5-af1e-409a-8c48-2e3245b1ef41 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.445126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafc8f5-af1e-409a-8c48-2e3245b1ef41 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.445562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafc8f5-af1e-409a-8c48-2e3245b1ef41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.469202 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-baafc8f5-af1e-409a-8c48-2e3245b1ef41 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:29.470640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-baafc8f5-af1e-409a-8c48-2e3245b1ef41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:29.471248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fdbe2-68c7-4ecc-8747-7106c1c14961 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:29.508178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4173d4-0090-4df6-ad0f-8b9ccb29f6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:29.509005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4173d4-0090-4df6-ad0f-8b9ccb29f6eb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.509475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4173d4-0090-4df6-ad0f-8b9ccb29f6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.509902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4173d4-0090-4df6-ad0f-8b9ccb29f6eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.521345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4173d4-0090-4df6-ad0f-8b9ccb29f6eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3d4173d4-0090-4df6-ad0f-8b9ccb29f6eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:29.530102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:29.531094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:29.531897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:29.534440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 136/272] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:29.535477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4173d4-0090-4df6-ad0f-8b9ccb29f6eb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:29.546050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 137/273] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:29.547100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafc8f5-af1e-409a-8c48-2e3245b1ef41 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:29.555436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:29.648015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4173d4-0090-4df6-ad0f-8b9ccb29f6eb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:29.651848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafc8f5-af1e-409a-8c48-2e3245b1ef41 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:29.663365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:29.686330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070c746-0ebe-45e3-8f58-db543ca74bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:29.687507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c746-0ebe-45e3-8f58-db543ca74bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.687989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c746-0ebe-45e3-8f58-db543ca74bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.688526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c746-0ebe-45e3-8f58-db543ca74bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.689696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070c746-0ebe-45e3-8f58-db543ca74bf4 None admin] RBAC: Adding query filter params (role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=46bc7b47b92148ffb91da999460d824f, user.id=660cf3927e2e440abdc14d58a574457e) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:29.689800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc2e3b-baa7-48a3-8c5e-ed4c9b016b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:29.690333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070c746-0ebe-45e3-8f58-db543ca74bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:29.690502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc2e3b-baa7-48a3-8c5e-ed4c9b016b3f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.690740 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc2e3b-baa7-48a3-8c5e-ed4c9b016b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.691012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc2e3b-baa7-48a3-8c5e-ed4c9b016b3f None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.691763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc2e3b-baa7-48a3-8c5e-ed4c9b016b3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:29.692067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc2e3b-baa7-48a3-8c5e-ed4c9b016b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:29.692984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070c746-0ebe-45e3-8f58-db543ca74bf4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:29.693487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc2e3b-baa7-48a3-8c5e-ed4c9b016b3f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:29.706805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 138/275] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:29.706805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc2e3b-baa7-48a3-8c5e-ed4c9b016b3f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:29.707854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 137/275] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:29 2022] GET /identity/v3/role_assignments?user.id=660cf3927e2e440abdc14d58a574457e&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:29.709254 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070c746-0ebe-45e3-8f58-db543ca74bf4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:29.789963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:29.815565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc2e3b-baa7-48a3-8c5e-ed4c9b016b3f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:29.857608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e69ffc-e6f6-4da1-87b9-cf231977b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:29.858686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e69ffc-e6f6-4da1-87b9-cf231977b17d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.858969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e69ffc-e6f6-4da1-87b9-cf231977b17d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.860121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e69ffc-e6f6-4da1-87b9-cf231977b17d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.862004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e69ffc-e6f6-4da1-87b9-cf231977b17d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:29.863475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e69ffc-e6f6-4da1-87b9-cf231977b17d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:29.865180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 139/276] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:29.870793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e69ffc-e6f6-4da1-87b9-cf231977b17d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:29.926443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070c746-0ebe-45e3-8f58-db543ca74bf4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:29.980085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac064d-eaee-4681-b85f-15b8c54fba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:29.980616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e69ffc-e6f6-4da1-87b9-cf231977b17d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:29.981646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac064d-eaee-4681-b85f-15b8c54fba8e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:29.982234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac064d-eaee-4681-b85f-15b8c54fba8e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:29.982726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac064d-eaee-4681-b85f-15b8c54fba8e None admin] PATH_INFO: `/v3/users/1d97250dfdc44e72bc3e5a22f5ed54fd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:29.999664 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac064d-eaee-4681-b85f-15b8c54fba8e None admin] RBAC: Authorizing `identity:get_user(user_id=1d97250dfdc44e72bc3e5a22f5ed54fd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:30.001177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac064d-eaee-4681-b85f-15b8c54fba8e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:30.003276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 138/277] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:29 2022] GET /identity/v3/users/1d97250dfdc44e72bc3e5a22f5ed54fd => generated 293 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:30.004537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c05c6-c70e-46b0-a667-73800b731066 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.004847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c05c6-c70e-46b0-a667-73800b731066 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.005125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c05c6-c70e-46b0-a667-73800b731066 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.006454 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 139/278] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:30.012426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937947-e767-4392-9714-2ddfa9f06efa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.012665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937947-e767-4392-9714-2ddfa9f06efa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.012975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937947-e767-4392-9714-2ddfa9f06efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.027203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c5c5e-f377-44a4-946c-6a669bafbf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:30.028263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c5c5e-f377-44a4-946c-6a669bafbf66 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.028552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c5c5e-f377-44a4-946c-6a669bafbf66 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.028863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c5c5e-f377-44a4-946c-6a669bafbf66 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.029588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c5c5e-f377-44a4-946c-6a669bafbf66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:30.031131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c5c5e-f377-44a4-946c-6a669bafbf66 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:30.032853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-dd937947-e767-4392-9714-2ddfa9f06efa None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:30.034268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dd937947-e767-4392-9714-2ddfa9f06efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:30.040324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:30.057720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 140/279] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:29 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:22:30.057720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c5c5e-f377-44a4-946c-6a669bafbf66 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:30.083936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:30.149015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 140/280] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:30.174237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd937947-e767-4392-9714-2ddfa9f06efa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:30.202962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c5c5e-f377-44a4-946c-6a669bafbf66 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:30.249762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230102c-f092-4d3e-a8e3-2d6e86d33290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:30.250429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230102c-f092-4d3e-a8e3-2d6e86d33290 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.250876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230102c-f092-4d3e-a8e3-2d6e86d33290 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.251160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230102c-f092-4d3e-a8e3-2d6e86d33290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.252594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230102c-f092-4d3e-a8e3-2d6e86d33290 None admin] RBAC: Adding query filter params (role.id=9ae7e8c79b724c70890cf352b9748307, user.id=1d97250dfdc44e72bc3e5a22f5ed54fd, scope.system=all) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:30.252943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230102c-f092-4d3e-a8e3-2d6e86d33290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:30.254886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230102c-f092-4d3e-a8e3-2d6e86d33290 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:30.260903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 141/281] 10.222.0.69 () {56 vars in 1308 bytes} [Fri Jun 24 10:22:30 2022] GET /identity/v3/role_assignments?user.id=1d97250dfdc44e72bc3e5a22f5ed54fd&scope.system=all&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:30.262175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb53fab-4f9e-476e-884d-9df595a4ab1b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.262498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb53fab-4f9e-476e-884d-9df595a4ab1b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.262781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb53fab-4f9e-476e-884d-9df595a4ab1b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.264047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 142/282] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:30.270438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ae0b4-87bb-4d5f-84c0-caaf74f4a2a5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.270744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ae0b4-87bb-4d5f-84c0-caaf74f4a2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.271004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ae0b4-87bb-4d5f-84c0-caaf74f4a2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.289161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8e0ae0b4-87bb-4d5f-84c0-caaf74f4a2a5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:30.290467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8e0ae0b4-87bb-4d5f-84c0-caaf74f4a2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:30.303173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd937947-e767-4392-9714-2ddfa9f06efa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:30.351512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cf599-37a5-4c43-8af1-d609964caa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:30.352609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cf599-37a5-4c43-8af1-d609964caa6c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.353146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cf599-37a5-4c43-8af1-d609964caa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.353585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cf599-37a5-4c43-8af1-d609964caa6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.361073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:30.368408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cf599-37a5-4c43-8af1-d609964caa6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6b4cf599-37a5-4c43-8af1-d609964caa6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:30.376979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:30.377914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:30.378678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:30.384858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 141/283] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:30.386722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cf599-37a5-4c43-8af1-d609964caa6c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:30.388258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 143/284] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:30.411247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ae0b4-87bb-4d5f-84c0-caaf74f4a2a5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:30.423484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:30.516513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cf599-37a5-4c43-8af1-d609964caa6c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:30.547167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ae0b4-87bb-4d5f-84c0-caaf74f4a2a5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:30.554071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088fae2-5abf-4d9a-b950-192777bd49cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:30.555001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088fae2-5abf-4d9a-b950-192777bd49cd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.555423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088fae2-5abf-4d9a-b950-192777bd49cd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.555866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088fae2-5abf-4d9a-b950-192777bd49cd None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.556699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088fae2-5abf-4d9a-b950-192777bd49cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:30.557074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088fae2-5abf-4d9a-b950-192777bd49cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:30.558490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088fae2-5abf-4d9a-b950-192777bd49cd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:30.568959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 142/285] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:30.574623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088fae2-5abf-4d9a-b950-192777bd49cd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:30.605938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7cfb5-4edf-485a-9a79-70ef27154d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:30.606746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7cfb5-4edf-485a-9a79-70ef27154d68 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.607075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7cfb5-4edf-485a-9a79-70ef27154d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.607352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7cfb5-4edf-485a-9a79-70ef27154d68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.627557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7cfb5-4edf-485a-9a79-70ef27154d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0de7cfb5-4edf-485a-9a79-70ef27154d68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:30.635852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:30.636698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:30.637505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:30.640463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 144/286] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:30.646767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7cfb5-4edf-485a-9a79-70ef27154d68 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:30.683755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:30.696489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088fae2-5abf-4d9a-b950-192777bd49cd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:30.746091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:30.762431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3231ec2-389c-4287-8226-ea774c79cee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:30.763327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3231ec2-389c-4287-8226-ea774c79cee0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.763610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3231ec2-389c-4287-8226-ea774c79cee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.763882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3231ec2-389c-4287-8226-ea774c79cee0 None admin] PATH_INFO: `/v3/users/5faaed96d4fa40a7af73c8137799c781` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.765245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3231ec2-389c-4287-8226-ea774c79cee0 None admin] RBAC: Authorizing `identity:get_user(user_id=5faaed96d4fa40a7af73c8137799c781)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:30.766871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3231ec2-389c-4287-8226-ea774c79cee0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:30.768684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 143/287] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:30 2022] GET /identity/v3/users/5faaed96d4fa40a7af73c8137799c781 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:30.780871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3231ec2-389c-4287-8226-ea774c79cee0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:30.857898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7cfb5-4edf-485a-9a79-70ef27154d68 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:30.905502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd789e19-bf42-4c94-96c4-da74c2ff02c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:30.906524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd789e19-bf42-4c94-96c4-da74c2ff02c8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:30.906980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd789e19-bf42-4c94-96c4-da74c2ff02c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:30.907425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd789e19-bf42-4c94-96c4-da74c2ff02c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:30.908382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd789e19-bf42-4c94-96c4-da74c2ff02c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:30.908883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd789e19-bf42-4c94-96c4-da74c2ff02c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:30.910621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd789e19-bf42-4c94-96c4-da74c2ff02c8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:30.921833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 145/288] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:30.929733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd789e19-bf42-4c94-96c4-da74c2ff02c8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:30.976639 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3231ec2-389c-4287-8226-ea774c79cee0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:31.045420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c4a9d-cbfd-442a-8f6c-8432b948f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:31.046384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c4a9d-cbfd-442a-8f6c-8432b948f444 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:31.046661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c4a9d-cbfd-442a-8f6c-8432b948f444 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:31.046930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c4a9d-cbfd-442a-8f6c-8432b948f444 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:31.049487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c4a9d-cbfd-442a-8f6c-8432b948f444 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:31.050957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c4a9d-cbfd-442a-8f6c-8432b948f444 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:31.052673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 144/289] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:30 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:31.062902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c4a9d-cbfd-442a-8f6c-8432b948f444 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:31.109156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:31.116224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd789e19-bf42-4c94-96c4-da74c2ff02c8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:31.154714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:31.173624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69b16f-9e51-4940-84dd-ea366c9b12d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:31.174884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69b16f-9e51-4940-84dd-ea366c9b12d8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:31.175229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69b16f-9e51-4940-84dd-ea366c9b12d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:31.175550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69b16f-9e51-4940-84dd-ea366c9b12d8 None admin] PATH_INFO: `/v3/users/6058a77019bd47ae89dcc6997ae81f08` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:31.176966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69b16f-9e51-4940-84dd-ea366c9b12d8 None admin] RBAC: Authorizing `identity:get_user(user_id=6058a77019bd47ae89dcc6997ae81f08)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:31.179115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69b16f-9e51-4940-84dd-ea366c9b12d8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:31.180849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 146/290] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:30 2022] GET /identity/v3/users/6058a77019bd47ae89dcc6997ae81f08 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:31.186938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69b16f-9e51-4940-84dd-ea366c9b12d8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:31.295791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69b16f-9e51-4940-84dd-ea366c9b12d8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:31.319306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c4a9d-cbfd-442a-8f6c-8432b948f444 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:31.359734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d94064-9204-4672-86d9-e40a97b4466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:31.360546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d94064-9204-4672-86d9-e40a97b4466d None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:31.360848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d94064-9204-4672-86d9-e40a97b4466d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:31.361127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d94064-9204-4672-86d9-e40a97b4466d None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f/users/5faaed96d4fa40a7af73c8137799c781/roles/c0d100940f624679824541cd7101a203` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:31.363727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d94064-9204-4672-86d9-e40a97b4466d None admin] RBAC: Authorizing `identity:create_grant(project_id=46bc7b47b92148ffb91da999460d824f, user_id=5faaed96d4fa40a7af73c8137799c781, role_id=c0d100940f624679824541cd7101a203)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:31.365201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d94064-9204-4672-86d9-e40a97b4466d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:31.368590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed88f6-d8db-4a67-83ae-45334c25d63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:31.369261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed88f6-d8db-4a67-83ae-45334c25d63e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:31.369541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed88f6-d8db-4a67-83ae-45334c25d63e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:31.369815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed88f6-d8db-4a67-83ae-45334c25d63e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:31.400646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 145/291] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:22:31 2022] PUT /identity/v3/projects/46bc7b47b92148ffb91da999460d824f/users/5faaed96d4fa40a7af73c8137799c781/roles/c0d100940f624679824541cd7101a203 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:31.446219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:31.475883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed88f6-d8db-4a67-83ae-45334c25d63e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:31.481623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed88f6-d8db-4a67-83ae-45334c25d63e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:31.635010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 147/292] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:22:31.640315 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d94064-9204-4672-86d9-e40a97b4466d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:31.750495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:31.766647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d94064-9204-4672-86d9-e40a97b4466d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:31.800756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fe8e6-bf0b-42bd-a815-2b9d656be022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:31.801641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fe8e6-bf0b-42bd-a815-2b9d656be022 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:31.801925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fe8e6-bf0b-42bd-a815-2b9d656be022 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:31.802207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fe8e6-bf0b-42bd-a815-2b9d656be022 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:31.803401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fe8e6-bf0b-42bd-a815-2b9d656be022 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:31.804946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fe8e6-bf0b-42bd-a815-2b9d656be022 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:31.806665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 146/293] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:31 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:31.812499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed88f6-d8db-4a67-83ae-45334c25d63e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:31.888772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:31.919432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed88f6-d8db-4a67-83ae-45334c25d63e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:31.953249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33b928-4901-4f80-9113-aecfffcc3f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:31.953903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33b928-4901-4f80-9113-aecfffcc3f23 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:31.954186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33b928-4901-4f80-9113-aecfffcc3f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:31.954794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33b928-4901-4f80-9113-aecfffcc3f23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:31.955623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33b928-4901-4f80-9113-aecfffcc3f23 None admin] RBAC: Adding query filter params (role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=46bc7b47b92148ffb91da999460d824f, user.id=6058a77019bd47ae89dcc6997ae81f08) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:31.955932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33b928-4901-4f80-9113-aecfffcc3f23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:31.957390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33b928-4901-4f80-9113-aecfffcc3f23 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:31.963332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 148/294] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:31 2022] GET /identity/v3/role_assignments?user.id=6058a77019bd47ae89dcc6997ae81f08&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:32.520129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afc3f3-b4c9-4030-80fe-0f35c061311a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.520471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afc3f3-b4c9-4030-80fe-0f35c061311a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.520693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afc3f3-b4c9-4030-80fe-0f35c061311a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.522370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 147/295] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:32.528564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4a887-d479-4cfa-828b-559a5cb88d8e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.529009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4a887-d479-4cfa-828b-559a5cb88d8e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.529294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4a887-d479-4cfa-828b-559a5cb88d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.546137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-51f4a887-d479-4cfa-828b-559a5cb88d8e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:32.547542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-51f4a887-d479-4cfa-828b-559a5cb88d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:32.619282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 149/296] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:32.628112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8438f-c287-444b-8dc7-757f7517ea5a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.628399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8438f-c287-444b-8dc7-757f7517ea5a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.628677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8438f-c287-444b-8dc7-757f7517ea5a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.630348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 148/297] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:32.635390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64966758-41dc-406e-87dc-8f60f507f219 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.635684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64966758-41dc-406e-87dc-8f60f507f219 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.635966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64966758-41dc-406e-87dc-8f60f507f219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.650069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8438f-c287-444b-8dc7-757f7517ea5a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:32.663974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-64966758-41dc-406e-87dc-8f60f507f219 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:32.665219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-64966758-41dc-406e-87dc-8f60f507f219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:32.737872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:32.743892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 150/298] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:32.745230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe0153-ff89-4918-a5c2-b802cde28949 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.745510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe0153-ff89-4918-a5c2-b802cde28949 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.745782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe0153-ff89-4918-a5c2-b802cde28949 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.747274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 151/299] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:32.754026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f80702-5c4a-422c-aeca-65e9c11a07a5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.754416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f80702-5c4a-422c-aeca-65e9c11a07a5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.754790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f80702-5c4a-422c-aeca-65e9c11a07a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.763195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8438f-c287-444b-8dc7-757f7517ea5a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:32.772142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-53f80702-5c4a-422c-aeca-65e9c11a07a5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:32.773369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-53f80702-5c4a-422c-aeca-65e9c11a07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:32.799005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870aa38-0687-4095-ae8d-f0d50f47b73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:32.800120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870aa38-0687-4095-ae8d-f0d50f47b73f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.800404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870aa38-0687-4095-ae8d-f0d50f47b73f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.800674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870aa38-0687-4095-ae8d-f0d50f47b73f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.811151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870aa38-0687-4095-ae8d-f0d50f47b73f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b870aa38-0687-4095-ae8d-f0d50f47b73f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:32.818796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:32.819668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:32.820435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 149/300] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:32.821951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870aa38-0687-4095-ae8d-f0d50f47b73f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:32.841034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:32.845356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 152/301] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:32.854021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f80702-5c4a-422c-aeca-65e9c11a07a5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:32.934679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870aa38-0687-4095-ae8d-f0d50f47b73f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:32.966047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f80702-5c4a-422c-aeca-65e9c11a07a5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:32.973910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4655e-84ef-4aed-af89-a4e7594a705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:32.974835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4655e-84ef-4aed-af89-a4e7594a705b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.975120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4655e-84ef-4aed-af89-a4e7594a705b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.975395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4655e-84ef-4aed-af89-a4e7594a705b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.976706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4655e-84ef-4aed-af89-a4e7594a705b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:32.978297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4655e-84ef-4aed-af89-a4e7594a705b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:32.984649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 150/302] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:32.984649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4655e-84ef-4aed-af89-a4e7594a705b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:33.002532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:33.007599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec301d-4557-403f-8bab-5d7ca3c8c538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.008282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec301d-4557-403f-8bab-5d7ca3c8c538 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.008565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec301d-4557-403f-8bab-5d7ca3c8c538 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.008880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec301d-4557-403f-8bab-5d7ca3c8c538 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.009644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec301d-4557-403f-8bab-5d7ca3c8c538 None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:33.009960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec301d-4557-403f-8bab-5d7ca3c8c538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:33.011345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec301d-4557-403f-8bab-5d7ca3c8c538 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:33.020284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 153/303] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:33.021078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec301d-4557-403f-8bab-5d7ca3c8c538 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:33.074814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:33.095878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4655e-84ef-4aed-af89-a4e7594a705b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:33.125190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec301d-4557-403f-8bab-5d7ca3c8c538 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:33.131607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55d539-4d88-4a90-9a43-a69171787813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.132830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55d539-4d88-4a90-9a43-a69171787813 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.133118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55d539-4d88-4a90-9a43-a69171787813 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.133393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55d539-4d88-4a90-9a43-a69171787813 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.144371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55d539-4d88-4a90-9a43-a69171787813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8a55d539-4d88-4a90-9a43-a69171787813 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:33.152619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:33.153506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:33.154277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 151/304] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:33.155030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55d539-4d88-4a90-9a43-a69171787813 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:33.168166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7426c-3004-4e40-a7c4-4e61f80eff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.168864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7426c-3004-4e40-a7c4-4e61f80eff32 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.169150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7426c-3004-4e40-a7c4-4e61f80eff32 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.169425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7426c-3004-4e40-a7c4-4e61f80eff32 None admin] PATH_INFO: `/v3/groups` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.170109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7426c-3004-4e40-a7c4-4e61f80eff32 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:33.171568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7426c-3004-4e40-a7c4-4e61f80eff32 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:33.187181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 154/305] 10.222.0.69 () {60 vars in 1134 bytes} [Fri Jun 24 10:22:33 2022] POST /identity/v3/groups => generated 229 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:33.188082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7426c-3004-4e40-a7c4-4e61f80eff32 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:33.265371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:33.285927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55d539-4d88-4a90-9a43-a69171787813 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:33.331317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1f9b7-ebd6-40e0-a80c-b869503ae526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.332561 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1f9b7-ebd6-40e0-a80c-b869503ae526 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.332932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1f9b7-ebd6-40e0-a80c-b869503ae526 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.333278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1f9b7-ebd6-40e0-a80c-b869503ae526 None admin] PATH_INFO: `/v3/users/1d97250dfdc44e72bc3e5a22f5ed54fd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.334603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7426c-3004-4e40-a7c4-4e61f80eff32 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:33.334712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1f9b7-ebd6-40e0-a80c-b869503ae526 None admin] RBAC: Authorizing `identity:get_user(user_id=1d97250dfdc44e72bc3e5a22f5ed54fd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:33.336226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1f9b7-ebd6-40e0-a80c-b869503ae526 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:33.339382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 152/306] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:33 2022] GET /identity/v3/users/1d97250dfdc44e72bc3e5a22f5ed54fd => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:33.346138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1f9b7-ebd6-40e0-a80c-b869503ae526 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:33.362635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:33.383551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729df80f-e9a4-4bef-8245-4580cc8200ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.384240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729df80f-e9a4-4bef-8245-4580cc8200ec None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.385534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729df80f-e9a4-4bef-8245-4580cc8200ec None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.385534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729df80f-e9a4-4bef-8245-4580cc8200ec None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.386264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729df80f-e9a4-4bef-8245-4580cc8200ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:33.387534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729df80f-e9a4-4bef-8245-4580cc8200ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:33.389936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729df80f-e9a4-4bef-8245-4580cc8200ec None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:33.398550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 155/307] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:33.407147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729df80f-e9a4-4bef-8245-4580cc8200ec None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:33.486882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1f9b7-ebd6-40e0-a80c-b869503ae526 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:33.523239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87962a70-d17e-44f7-9b59-e88a3c3696d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.524460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87962a70-d17e-44f7-9b59-e88a3c3696d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.524984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87962a70-d17e-44f7-9b59-e88a3c3696d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.525522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87962a70-d17e-44f7-9b59-e88a3c3696d6 None admin] PATH_INFO: `/v3/system/users/1d97250dfdc44e72bc3e5a22f5ed54fd/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.527198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87962a70-d17e-44f7-9b59-e88a3c3696d6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1d97250dfdc44e72bc3e5a22f5ed54fd, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:33.528860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87962a70-d17e-44f7-9b59-e88a3c3696d6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:33.536857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729df80f-e9a4-4bef-8245-4580cc8200ec None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:33.542428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 153/308] 10.222.0.69 () {58 vars in 1334 bytes} [Fri Jun 24 10:22:33 2022] PUT /identity/v3/system/users/1d97250dfdc44e72bc3e5a22f5ed54fd/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:22:33.553566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:33.575333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f500c-a41d-4d13-b7a3-b93dbd802ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.576257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f500c-a41d-4d13-b7a3-b93dbd802ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.576537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f500c-a41d-4d13-b7a3-b93dbd802ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.576893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f500c-a41d-4d13-b7a3-b93dbd802ad0 None admin] PATH_INFO: `/v3/users/a26c236d252847eebf01b34bd3c3be39` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.594210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f500c-a41d-4d13-b7a3-b93dbd802ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=a26c236d252847eebf01b34bd3c3be39)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:33.595716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f500c-a41d-4d13-b7a3-b93dbd802ad0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:33.597902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 156/309] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:33 2022] GET /identity/v3/users/a26c236d252847eebf01b34bd3c3be39 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:33.604305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87962a70-d17e-44f7-9b59-e88a3c3696d6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:33.685200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:33.718875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87962a70-d17e-44f7-9b59-e88a3c3696d6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:33.753862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80137d4-e65e-4887-80d4-fa39f654d029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.754532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80137d4-e65e-4887-80d4-fa39f654d029 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.754815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80137d4-e65e-4887-80d4-fa39f654d029 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.755088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80137d4-e65e-4887-80d4-fa39f654d029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.755929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80137d4-e65e-4887-80d4-fa39f654d029 None admin] RBAC: Adding query filter params (role.id=c0d100940f624679824541cd7101a203, user.id=a26c236d252847eebf01b34bd3c3be39, scope.system=all) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:33.756242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80137d4-e65e-4887-80d4-fa39f654d029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:33.757701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80137d4-e65e-4887-80d4-fa39f654d029 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:33.763220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 154/310] 10.222.0.69 () {56 vars in 1308 bytes} [Fri Jun 24 10:22:33 2022] GET /identity/v3/role_assignments?user.id=a26c236d252847eebf01b34bd3c3be39&scope.system=all&role.id=c0d100940f624679824541cd7101a203 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:33.855072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9802abe-b73a-4fe6-b345-74decc6049d0 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.855567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9802abe-b73a-4fe6-b345-74decc6049d0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.855567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9802abe-b73a-4fe6-b345-74decc6049d0 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.857251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 157/311] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:33.863332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc694-5b5f-45d2-a5e1-57c0aa85e891 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.863624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc694-5b5f-45d2-a5e1-57c0aa85e891 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.863893 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc694-5b5f-45d2-a5e1-57c0aa85e891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.882171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-01adc694-5b5f-45d2-a5e1-57c0aa85e891 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:33.883382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-01adc694-5b5f-45d2-a5e1-57c0aa85e891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:33.956050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 155/312] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:33.978840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9802abe-b73a-4fe6-b345-74decc6049d0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:34.083097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9802abe-b73a-4fe6-b345-74decc6049d0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:34.117555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f4664-14e8-415b-874d-1ec3c5b3c991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:34.118275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f4664-14e8-415b-874d-1ec3c5b3c991 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:34.118550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f4664-14e8-415b-874d-1ec3c5b3c991 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:34.118816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f4664-14e8-415b-874d-1ec3c5b3c991 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:34.129366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f4664-14e8-415b-874d-1ec3c5b3c991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-df3f4664-14e8-415b-874d-1ec3c5b3c991 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:34.137213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:34.138063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:34.138831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 158/313] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:34.144579 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc694-5b5f-45d2-a5e1-57c0aa85e891 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:34.211562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:34.248256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc694-5b5f-45d2-a5e1-57c0aa85e891 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:34.285000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db3308-ea9e-4708-ba04-cbfe9dbc0efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:34.285676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db3308-ea9e-4708-ba04-cbfe9dbc0efa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:34.285955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db3308-ea9e-4708-ba04-cbfe9dbc0efa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:34.286228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db3308-ea9e-4708-ba04-cbfe9dbc0efa None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:34.286981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db3308-ea9e-4708-ba04-cbfe9dbc0efa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:34.287284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db3308-ea9e-4708-ba04-cbfe9dbc0efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:34.288657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db3308-ea9e-4708-ba04-cbfe9dbc0efa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:34.297224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 156/314] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:34.302697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f4664-14e8-415b-874d-1ec3c5b3c991 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:34.338658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:34.409149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f4664-14e8-415b-874d-1ec3c5b3c991 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:34.450396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c5000-feff-464c-b510-cf85fdf82706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:34.451289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c5000-feff-464c-b510-cf85fdf82706 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:34.451597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c5000-feff-464c-b510-cf85fdf82706 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:34.452045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c5000-feff-464c-b510-cf85fdf82706 None admin] PATH_INFO: `/v3/users/5faaed96d4fa40a7af73c8137799c781` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:34.453281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c5000-feff-464c-b510-cf85fdf82706 None admin] RBAC: Authorizing `identity:get_user(user_id=5faaed96d4fa40a7af73c8137799c781)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:34.455076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c5000-feff-464c-b510-cf85fdf82706 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:34.456905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 159/315] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:34 2022] GET /identity/v3/users/5faaed96d4fa40a7af73c8137799c781 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:34.462170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db3308-ea9e-4708-ba04-cbfe9dbc0efa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:34.569969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db3308-ea9e-4708-ba04-cbfe9dbc0efa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:34.597773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:34.606643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99517a88-a289-4ac1-95bc-f65757b9cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:34.607567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99517a88-a289-4ac1-95bc-f65757b9cedf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:34.608226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99517a88-a289-4ac1-95bc-f65757b9cedf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:34.608726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99517a88-a289-4ac1-95bc-f65757b9cedf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:34.678009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99517a88-a289-4ac1-95bc-f65757b9cedf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:34.679792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99517a88-a289-4ac1-95bc-f65757b9cedf None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:34.786120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 157/316] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:34.794669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c5000-feff-464c-b510-cf85fdf82706 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:34.889533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:34.893081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c5000-feff-464c-b510-cf85fdf82706 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:34.929694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9d48e-682a-4a64-80b3-4543321e62e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:34.930478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9d48e-682a-4a64-80b3-4543321e62e5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:34.930760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9d48e-682a-4a64-80b3-4543321e62e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:34.931033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9d48e-682a-4a64-80b3-4543321e62e5 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:34.932214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9d48e-682a-4a64-80b3-4543321e62e5 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:34.933689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9d48e-682a-4a64-80b3-4543321e62e5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:34.935794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 160/317] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:34 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:34.941559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99517a88-a289-4ac1-95bc-f65757b9cedf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:34.970412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:35.041841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99517a88-a289-4ac1-95bc-f65757b9cedf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:35.076080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953b363-0a67-4913-872b-f1942e0d087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:35.076948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b363-0a67-4913-872b-f1942e0d087f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.077237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b363-0a67-4913-872b-f1942e0d087f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.077512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b363-0a67-4913-872b-f1942e0d087f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.078345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953b363-0a67-4913-872b-f1942e0d087f None admin] RBAC: Adding query filter params (role.id=c0d100940f624679824541cd7101a203, scope.project.id=46bc7b47b92148ffb91da999460d824f, user.id=5faaed96d4fa40a7af73c8137799c781) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:35.078652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953b363-0a67-4913-872b-f1942e0d087f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:35.080411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953b363-0a67-4913-872b-f1942e0d087f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:35.086740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 158/318] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:22:34 2022] GET /identity/v3/role_assignments?user.id=5faaed96d4fa40a7af73c8137799c781&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=c0d100940f624679824541cd7101a203 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:35.369142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49b1e9-fe36-43c3-b54e-054df4369e87 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.369339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49b1e9-fe36-43c3-b54e-054df4369e87 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.369590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49b1e9-fe36-43c3-b54e-054df4369e87 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.370972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 161/319] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:35.376615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e77c56-85c2-4f08-a435-6e56ee484d1d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.376905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e77c56-85c2-4f08-a435-6e56ee484d1d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.377206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e77c56-85c2-4f08-a435-6e56ee484d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.394668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-20e77c56-85c2-4f08-a435-6e56ee484d1d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:35.395863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-20e77c56-85c2-4f08-a435-6e56ee484d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:35.470958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 159/320] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:35.493421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49b1e9-fe36-43c3-b54e-054df4369e87 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:35.600421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49b1e9-fe36-43c3-b54e-054df4369e87 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:35.612471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549f2ab-d7a8-4fb1-8899-7dfe03ca7904 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.612766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549f2ab-d7a8-4fb1-8899-7dfe03ca7904 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.613070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549f2ab-d7a8-4fb1-8899-7dfe03ca7904 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.614440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 160/321] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:35.619537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409576c5-9402-4a84-99f5-5b0b2059ef1f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.619769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409576c5-9402-4a84-99f5-5b0b2059ef1f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.620044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409576c5-9402-4a84-99f5-5b0b2059ef1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.634480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee05b3d-86aa-4e03-8e7c-b1616691b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:35.635272 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05b3d-86aa-4e03-8e7c-b1616691b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.635551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05b3d-86aa-4e03-8e7c-b1616691b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.635822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05b3d-86aa-4e03-8e7c-b1616691b3e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.636949 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-409576c5-9402-4a84-99f5-5b0b2059ef1f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:35.637049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee05b3d-86aa-4e03-8e7c-b1616691b3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:35.638133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-409576c5-9402-4a84-99f5-5b0b2059ef1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:35.638425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee05b3d-86aa-4e03-8e7c-b1616691b3e0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:35.644444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 162/322] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:35.645485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee05b3d-86aa-4e03-8e7c-b1616691b3e0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:35.656407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:35.685245 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:35.713816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 161/323] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:35.737198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409576c5-9402-4a84-99f5-5b0b2059ef1f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:35.767421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee05b3d-86aa-4e03-8e7c-b1616691b3e0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:35.818907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4db82b-ba85-4d29-954c-71dc377d57ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:35.819989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db82b-ba85-4d29-954c-71dc377d57ea None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.820462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db82b-ba85-4d29-954c-71dc377d57ea None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.820914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db82b-ba85-4d29-954c-71dc377d57ea None admin] PATH_INFO: `/v3/groups` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.822103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4db82b-ba85-4d29-954c-71dc377d57ea None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:35.824557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4db82b-ba85-4d29-954c-71dc377d57ea None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:35.843763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 163/324] 10.222.0.69 () {60 vars in 1134 bytes} [Fri Jun 24 10:22:35 2022] POST /identity/v3/groups => generated 226 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:22:35.879323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b0d69-b0a2-458f-bedc-5494db844fb4 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.880124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b0d69-b0a2-458f-bedc-5494db844fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.880436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b0d69-b0a2-458f-bedc-5494db844fb4 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.882890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 164/325] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:35.883721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409576c5-9402-4a84-99f5-5b0b2059ef1f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:35.890036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b71ed-0e3b-4bd0-aae5-1fd04990c97d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.890549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b71ed-0e3b-4bd0-aae5-1fd04990c97d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.890620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b71ed-0e3b-4bd0-aae5-1fd04990c97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.911348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7a9b71ed-0e3b-4bd0-aae5-1fd04990c97d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:35.912390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7a9b71ed-0e3b-4bd0-aae5-1fd04990c97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:35.929498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7128e-948e-4679-9154-49c19506c057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:35.930333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7128e-948e-4679-9154-49c19506c057 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.930615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7128e-948e-4679-9154-49c19506c057 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.930883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7128e-948e-4679-9154-49c19506c057 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.942537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7128e-948e-4679-9154-49c19506c057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-cdf7128e-948e-4679-9154-49c19506c057 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:35.950460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:35.951402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:35.957723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 162/326] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:35.960012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7128e-948e-4679-9154-49c19506c057 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:35.976090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:35.999588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 165/327] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:36.021747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b71ed-0e3b-4bd0-aae5-1fd04990c97d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:36.055516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:36.075004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7128e-948e-4679-9154-49c19506c057 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:36.119033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272334c0-e852-428f-9644-f5ac4dd40053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:36.120028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272334c0-e852-428f-9644-f5ac4dd40053 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:36.120470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272334c0-e852-428f-9644-f5ac4dd40053 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:36.120783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272334c0-e852-428f-9644-f5ac4dd40053 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:36.121560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272334c0-e852-428f-9644-f5ac4dd40053 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:36.121871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272334c0-e852-428f-9644-f5ac4dd40053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:36.123276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272334c0-e852-428f-9644-f5ac4dd40053 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:36.134471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 163/328] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:36.139858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272334c0-e852-428f-9644-f5ac4dd40053 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:36.148080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b71ed-0e3b-4bd0-aae5-1fd04990c97d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:36.192016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6f044-43d7-4c25-829b-845200b7a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:36.193260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6f044-43d7-4c25-829b-845200b7a792 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:36.193547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6f044-43d7-4c25-829b-845200b7a792 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:36.193824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6f044-43d7-4c25-829b-845200b7a792 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:36.209464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6f044-43d7-4c25-829b-845200b7a792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3ff6f044-43d7-4c25-829b-845200b7a792 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:36.218607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:36.219479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:36.220268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 166/329] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:36.226354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6f044-43d7-4c25-829b-845200b7a792 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:36.238972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:36.277532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272334c0-e852-428f-9644-f5ac4dd40053 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:36.315564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05a28-8ff7-496f-b8b1-b0f116c3daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:36.316768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05a28-8ff7-496f-b8b1-b0f116c3daf1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:36.317253 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05a28-8ff7-496f-b8b1-b0f116c3daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:36.317526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05a28-8ff7-496f-b8b1-b0f116c3daf1 None admin] PATH_INFO: `/v3/users/1d97250dfdc44e72bc3e5a22f5ed54fd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:36.318790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc05a28-8ff7-496f-b8b1-b0f116c3daf1 None admin] RBAC: Authorizing `identity:get_user(user_id=1d97250dfdc44e72bc3e5a22f5ed54fd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:36.320242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc05a28-8ff7-496f-b8b1-b0f116c3daf1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:36.326747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 164/330] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:36 2022] GET /identity/v3/users/1d97250dfdc44e72bc3e5a22f5ed54fd => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:36.328811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05a28-8ff7-496f-b8b1-b0f116c3daf1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:36.342628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:36.357576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6f044-43d7-4c25-829b-845200b7a792 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:36.402363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b2f12-1e95-440a-b452-68ad5d2ade56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:36.403418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b2f12-1e95-440a-b452-68ad5d2ade56 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:36.404059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b2f12-1e95-440a-b452-68ad5d2ade56 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:36.404551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b2f12-1e95-440a-b452-68ad5d2ade56 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:36.406213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b2f12-1e95-440a-b452-68ad5d2ade56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:36.406805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b2f12-1e95-440a-b452-68ad5d2ade56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:36.408938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b2f12-1e95-440a-b452-68ad5d2ade56 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:36.419920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 167/331] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:36.425097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b2f12-1e95-440a-b452-68ad5d2ade56 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:36.487557 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05a28-8ff7-496f-b8b1-b0f116c3daf1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:36.535623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c4dec-6912-4fe1-a48f-ea354f7a2420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:36.537247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c4dec-6912-4fe1-a48f-ea354f7a2420 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:36.537674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c4dec-6912-4fe1-a48f-ea354f7a2420 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:36.538090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c4dec-6912-4fe1-a48f-ea354f7a2420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:36.539721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c4dec-6912-4fe1-a48f-ea354f7a2420 None admin] RBAC: Adding query filter params (role.id=9ae7e8c79b724c70890cf352b9748307, user.id=1d97250dfdc44e72bc3e5a22f5ed54fd, scope.system=all) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:36.540437 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c4dec-6912-4fe1-a48f-ea354f7a2420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:36.542077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c4dec-6912-4fe1-a48f-ea354f7a2420 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:36.542375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b2f12-1e95-440a-b452-68ad5d2ade56 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:36.554696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:36.558883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 165/332] 10.222.0.69 () {56 vars in 1308 bytes} [Fri Jun 24 10:22:36 2022] GET /identity/v3/role_assignments?user.id=1d97250dfdc44e72bc3e5a22f5ed54fd&scope.system=all&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:36.595729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74021eb5-13a5-4a30-8373-89ab2db38004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:36.597064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74021eb5-13a5-4a30-8373-89ab2db38004 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:36.597571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74021eb5-13a5-4a30-8373-89ab2db38004 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:36.598051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74021eb5-13a5-4a30-8373-89ab2db38004 None admin] PATH_INFO: `/v3/users/a26c236d252847eebf01b34bd3c3be39` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:36.599582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74021eb5-13a5-4a30-8373-89ab2db38004 None admin] RBAC: Authorizing `identity:get_user(user_id=a26c236d252847eebf01b34bd3c3be39)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:36.601325 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74021eb5-13a5-4a30-8373-89ab2db38004 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:36.603631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 168/333] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:36 2022] GET /identity/v3/users/a26c236d252847eebf01b34bd3c3be39 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:36.609140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c4dec-6912-4fe1-a48f-ea354f7a2420 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:36.745298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c4dec-6912-4fe1-a48f-ea354f7a2420 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:36.787272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca049b-998e-4be2-a956-b8e9efa59535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:36.788160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca049b-998e-4be2-a956-b8e9efa59535 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:36.788443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca049b-998e-4be2-a956-b8e9efa59535 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:36.788725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca049b-998e-4be2-a956-b8e9efa59535 None admin] PATH_INFO: `/v3/system/users/a26c236d252847eebf01b34bd3c3be39/roles/c0d100940f624679824541cd7101a203` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:36.790246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca049b-998e-4be2-a956-b8e9efa59535 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a26c236d252847eebf01b34bd3c3be39, role_id=c0d100940f624679824541cd7101a203)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:36.791635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca049b-998e-4be2-a956-b8e9efa59535 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:36.808270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 166/334] 10.222.0.69 () {58 vars in 1334 bytes} [Fri Jun 24 10:22:36 2022] PUT /identity/v3/system/users/a26c236d252847eebf01b34bd3c3be39/roles/c0d100940f624679824541cd7101a203 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:38.401042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7f3f5-06e4-4ec3-b5d7-a7779975fc20 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:38.402086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7f3f5-06e4-4ec3-b5d7-a7779975fc20 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:38.402584 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7f3f5-06e4-4ec3-b5d7-a7779975fc20 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:38.404167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 169/335] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:38.409648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e1c7c-8629-474f-9969-913337c6ae82 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:38.410493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e1c7c-8629-474f-9969-913337c6ae82 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:38.410962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e1c7c-8629-474f-9969-913337c6ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:38.433740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f59e1c7c-8629-474f-9969-913337c6ae82 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:38.435881 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f59e1c7c-8629-474f-9969-913337c6ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:38.468981 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27deb34-5030-4bd6-aa5d-a392d0d4a878 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:38.469297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27deb34-5030-4bd6-aa5d-a392d0d4a878 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:38.469535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27deb34-5030-4bd6-aa5d-a392d0d4a878 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:38.471171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 170/336] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:38.486240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8577ed-d1e6-4bd9-98c4-a1785881a5d1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:38.486512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8577ed-d1e6-4bd9-98c4-a1785881a5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:38.486788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8577ed-d1e6-4bd9-98c4-a1785881a5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:38.513099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9b8577ed-d1e6-4bd9-98c4-a1785881a5d1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:38.514333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9b8577ed-d1e6-4bd9-98c4-a1785881a5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:38.532539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:38.542751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:38.547784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 167/337] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:38.583549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e1c7c-8629-474f-9969-913337c6ae82 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:38.663335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 171/338] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:38.700793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8577ed-d1e6-4bd9-98c4-a1785881a5d1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:38.784600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e1c7c-8629-474f-9969-913337c6ae82 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:38.870165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4c449-86c1-47e4-a092-2f47ca6ec70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:38.871004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4c449-86c1-47e4-a092-2f47ca6ec70a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:38.871287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4c449-86c1-47e4-a092-2f47ca6ec70a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:38.871565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4c449-86c1-47e4-a092-2f47ca6ec70a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:38.909539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4c449-86c1-47e4-a092-2f47ca6ec70a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a7b4c449-86c1-47e4-a092-2f47ca6ec70a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:38.917725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:38.918608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:38.919803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 168/339] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:38.920468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abaa57-d88f-45ab-9738-bf0a3a171400 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:38.920468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abaa57-d88f-45ab-9738-bf0a3a171400 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:38.920852 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abaa57-d88f-45ab-9738-bf0a3a171400 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:38.922241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 169/340] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:38.928100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b4d2-ef90-4700-afdd-445fe0706faa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:38.928390 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b4d2-ef90-4700-afdd-445fe0706faa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:38.928666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b4d2-ef90-4700-afdd-445fe0706faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:38.947388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6c40b4d2-ef90-4700-afdd-445fe0706faa None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:38.948580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6c40b4d2-ef90-4700-afdd-445fe0706faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:38.958672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8577ed-d1e6-4bd9-98c4-a1785881a5d1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:39.003388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:39.023729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 170/341] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:39.024577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b4d2-ef90-4700-afdd-445fe0706faa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:39.043632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bcd40-d1aa-40d9-b848-b0abde41af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:39.044451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bcd40-d1aa-40d9-b848-b0abde41af26 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:39.044727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bcd40-d1aa-40d9-b848-b0abde41af26 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:39.045024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bcd40-d1aa-40d9-b848-b0abde41af26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:39.066918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bcd40-d1aa-40d9-b848-b0abde41af26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-2c7bcd40-d1aa-40d9-b848-b0abde41af26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:39.075490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:39.076339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:39.077699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 172/342] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:39.078335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bcd40-d1aa-40d9-b848-b0abde41af26 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:39.092944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:39.180843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b4d2-ef90-4700-afdd-445fe0706faa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:39.219283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94c3b8-8132-448b-bf43-5ec7111ada20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:39.219960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94c3b8-8132-448b-bf43-5ec7111ada20 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:39.220242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94c3b8-8132-448b-bf43-5ec7111ada20 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:39.220517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94c3b8-8132-448b-bf43-5ec7111ada20 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:39.221308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94c3b8-8132-448b-bf43-5ec7111ada20 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:39.221616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94c3b8-8132-448b-bf43-5ec7111ada20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:39.223009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94c3b8-8132-448b-bf43-5ec7111ada20 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:39.239078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bcd40-d1aa-40d9-b848-b0abde41af26 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:39.241637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 171/343] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:39.242474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94c3b8-8132-448b-bf43-5ec7111ada20 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:39.307812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2c398-179c-4b83-b1d0-ef175e22517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:39.308621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2c398-179c-4b83-b1d0-ef175e22517e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:39.308929 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2c398-179c-4b83-b1d0-ef175e22517e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:39.309205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2c398-179c-4b83-b1d0-ef175e22517e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:39.320312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:39.333886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2c398-179c-4b83-b1d0-ef175e22517e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ece2c398-179c-4b83-b1d0-ef175e22517e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:39.344347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:39.345307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:39.346085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:39.353537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 173/344] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:39.354395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2c398-179c-4b83-b1d0-ef175e22517e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:39.409724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94c3b8-8132-448b-bf43-5ec7111ada20 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:39.469488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde8f2b-a8ab-4613-96dc-67177e207d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:39.470183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde8f2b-a8ab-4613-96dc-67177e207d85 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:39.470465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde8f2b-a8ab-4613-96dc-67177e207d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:39.470763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde8f2b-a8ab-4613-96dc-67177e207d85 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:39.471513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde8f2b-a8ab-4613-96dc-67177e207d85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:39.471820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde8f2b-a8ab-4613-96dc-67177e207d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:39.473862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:39.474287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde8f2b-a8ab-4613-96dc-67177e207d85 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:39.482278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 172/345] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:39.483302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde8f2b-a8ab-4613-96dc-67177e207d85 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:39.515998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2c398-179c-4b83-b1d0-ef175e22517e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:39.562614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3a76-c50b-45ae-9a07-1ab8936bfec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:39.563649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3a76-c50b-45ae-9a07-1ab8936bfec8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:39.564127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3a76-c50b-45ae-9a07-1ab8936bfec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:39.564595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3a76-c50b-45ae-9a07-1ab8936bfec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:39.612860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde8f2b-a8ab-4613-96dc-67177e207d85 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:39.623589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:39.654207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb62f5-a7fc-460b-8480-aec0dfb9bb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:39.654961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb62f5-a7fc-460b-8480-aec0dfb9bb7a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:39.655255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb62f5-a7fc-460b-8480-aec0dfb9bb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:39.655532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb62f5-a7fc-460b-8480-aec0dfb9bb7a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:39.656299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb62f5-a7fc-460b-8480-aec0dfb9bb7a None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:39.656607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb62f5-a7fc-460b-8480-aec0dfb9bb7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:39.658134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb62f5-a7fc-460b-8480-aec0dfb9bb7a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:39.661143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d3a76-c50b-45ae-9a07-1ab8936bfec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:39.662828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d3a76-c50b-45ae-9a07-1ab8936bfec8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:39.672455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 173/346] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:39.672761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb62f5-a7fc-460b-8480-aec0dfb9bb7a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:39.733407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:39.775314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 174/347] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:39.781712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3a76-c50b-45ae-9a07-1ab8936bfec8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:39.861282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb62f5-a7fc-460b-8480-aec0dfb9bb7a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:39.930467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7fcc3-c684-4a17-9250-9ef991f195ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:39.931272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7fcc3-c684-4a17-9250-9ef991f195ce None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:39.931555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7fcc3-c684-4a17-9250-9ef991f195ce None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:39.931835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7fcc3-c684-4a17-9250-9ef991f195ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:39.951291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3a76-c50b-45ae-9a07-1ab8936bfec8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:39.989499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdca4f-e9d8-440b-ac04-04520d18cdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:39.990038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:39.990144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdca4f-e9d8-440b-ac04-04520d18cdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:39.990293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdca4f-e9d8-440b-ac04-04520d18cdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:39.990538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdca4f-e9d8-440b-ac04-04520d18cdd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.020422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7fcc3-c684-4a17-9250-9ef991f195ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:40.021945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7fcc3-c684-4a17-9250-9ef991f195ce None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:40.031389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:40.078019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdca4f-e9d8-440b-ac04-04520d18cdd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:40.079878 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdca4f-e9d8-440b-ac04-04520d18cdd0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:40.145955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 174/348] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:40.146554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7fcc3-c684-4a17-9250-9ef991f195ce None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:40.192378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 175/349] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:40.195222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdc953-46a4-4cdc-949e-4d207642b5f1 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.195222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdc953-46a4-4cdc-949e-4d207642b5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.195222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdc953-46a4-4cdc-949e-4d207642b5f1 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.196141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 176/350] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:40.197931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f26d62-0673-485f-9dc7-b9247e1c4d41 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.197931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f26d62-0673-485f-9dc7-b9247e1c4d41 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.197931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f26d62-0673-485f-9dc7-b9247e1c4d41 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.199544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 177/351] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:40.200355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f26d62-0673-485f-9dc7-b9247e1c4d41 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:40.245709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:40.293986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:40.298066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7fcc3-c684-4a17-9250-9ef991f195ce None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:40.344307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f26d62-0673-485f-9dc7-b9247e1c4d41 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:40.349341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e96787-e38a-4964-bc47-19045537e9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:40.350237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e96787-e38a-4964-bc47-19045537e9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.350519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e96787-e38a-4964-bc47-19045537e9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.350794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e96787-e38a-4964-bc47-19045537e9d9 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.352520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e96787-e38a-4964-bc47-19045537e9d9 None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:40.354066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e96787-e38a-4964-bc47-19045537e9d9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:40.356332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 175/352] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:40 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:40.357222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e96787-e38a-4964-bc47-19045537e9d9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:40.395338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbdbdd-0e6c-4aee-9a73-13c3173055c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:40.396267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbdbdd-0e6c-4aee-9a73-13c3173055c2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.396594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbdbdd-0e6c-4aee-9a73-13c3173055c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.396967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbdbdd-0e6c-4aee-9a73-13c3173055c2 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.398154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbdbdd-0e6c-4aee-9a73-13c3173055c2 None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:40.399729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbdbdd-0e6c-4aee-9a73-13c3173055c2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:40.401559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 178/353] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:40 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:40.403733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095b54e-95e9-4895-9904-468120a5eec8 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.403733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095b54e-95e9-4895-9904-468120a5eec8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.403733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095b54e-95e9-4895-9904-468120a5eec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.423149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b095b54e-95e9-4895-9904-468120a5eec8 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:40.424463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b095b54e-95e9-4895-9904-468120a5eec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:40.486054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e96787-e38a-4964-bc47-19045537e9d9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:40.511818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 179/354] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:40.511818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ff053-d5c8-4a0f-9266-f691b0f87a1a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.512340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ff053-d5c8-4a0f-9266-f691b0f87a1a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.512407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ff053-d5c8-4a0f-9266-f691b0f87a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.532858 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-dc6ff053-d5c8-4a0f-9266-f691b0f87a1a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:40.536334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dc6ff053-d5c8-4a0f-9266-f691b0f87a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:40.543298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833071f-e4f6-40e5-9dec-b7d0b3821930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:40.543298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833071f-e4f6-40e5-9dec-b7d0b3821930 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.543298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833071f-e4f6-40e5-9dec-b7d0b3821930 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.543298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833071f-e4f6-40e5-9dec-b7d0b3821930 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.544603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833071f-e4f6-40e5-9dec-b7d0b3821930 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:40.546212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833071f-e4f6-40e5-9dec-b7d0b3821930 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:40.547977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 176/355] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:40 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:40.548681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833071f-e4f6-40e5-9dec-b7d0b3821930 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:40.563916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:40.618044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:40.630204 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 180/356] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:40.633386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ff053-d5c8-4a0f-9266-f691b0f87a1a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:40.688262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833071f-e4f6-40e5-9dec-b7d0b3821930 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:40.733290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59439e-0356-4ab9-81b3-95b5e4d015a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:40.734256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59439e-0356-4ab9-81b3-95b5e4d015a8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.734535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59439e-0356-4ab9-81b3-95b5e4d015a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.734812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59439e-0356-4ab9-81b3-95b5e4d015a8 None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.742545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59439e-0356-4ab9-81b3-95b5e4d015a8 None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:40.744809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59439e-0356-4ab9-81b3-95b5e4d015a8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:40.747573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 177/357] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:40 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:40.749633 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59439e-0356-4ab9-81b3-95b5e4d015a8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:40.752095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ff053-d5c8-4a0f-9266-f691b0f87a1a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:40.788636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3bb3c-eed7-4565-bee8-1f43c6b2f1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:40.789456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3bb3c-eed7-4565-bee8-1f43c6b2f1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.789736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3bb3c-eed7-4565-bee8-1f43c6b2f1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.792798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3bb3c-eed7-4565-bee8-1f43c6b2f1d1 None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.792798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3bb3c-eed7-4565-bee8-1f43c6b2f1d1 None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:40.792798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3bb3c-eed7-4565-bee8-1f43c6b2f1d1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:40.794426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 181/358] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:40 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:40.795024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3bb3c-eed7-4565-bee8-1f43c6b2f1d1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:40.830651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:40.867588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:40.868895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59439e-0356-4ab9-81b3-95b5e4d015a8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:40.911087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171f4b0-9816-4351-bfd8-7fae632c5096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:40.911906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171f4b0-9816-4351-bfd8-7fae632c5096 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.912186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171f4b0-9816-4351-bfd8-7fae632c5096 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.912458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171f4b0-9816-4351-bfd8-7fae632c5096 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.918682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3bb3c-eed7-4565-bee8-1f43c6b2f1d1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:40.925658 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171f4b0-9816-4351-bfd8-7fae632c5096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8171f4b0-9816-4351-bfd8-7fae632c5096 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:40.937062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:40.938143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:40.939106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:40.941211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 178/359] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:40.942144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171f4b0-9816-4351-bfd8-7fae632c5096 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:40.968660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facda09f-4f55-4d2b-99df-5fccc2f14b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:40.969901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facda09f-4f55-4d2b-99df-5fccc2f14b68 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.969901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facda09f-4f55-4d2b-99df-5fccc2f14b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.970105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facda09f-4f55-4d2b-99df-5fccc2f14b68 None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.971284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facda09f-4f55-4d2b-99df-5fccc2f14b68 None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:40.972777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facda09f-4f55-4d2b-99df-5fccc2f14b68 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:40.974562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 182/360] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:40 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:40.976072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a80fd-3aec-4f19-8823-59c27b5e448a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:40.976072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a80fd-3aec-4f19-8823-59c27b5e448a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:40.976072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a80fd-3aec-4f19-8823-59c27b5e448a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:40.977745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 183/361] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:40.978297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a80fd-3aec-4f19-8823-59c27b5e448a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:41.064304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171f4b0-9816-4351-bfd8-7fae632c5096 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:41.099716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:41.109557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a80fd-3aec-4f19-8823-59c27b5e448a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:41.110886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417a96a-edb7-4af2-8a95-2cfe5cda82e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:41.111138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417a96a-edb7-4af2-8a95-2cfe5cda82e8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:41.111442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417a96a-edb7-4af2-8a95-2cfe5cda82e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:41.111722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417a96a-edb7-4af2-8a95-2cfe5cda82e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:41.127342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417a96a-edb7-4af2-8a95-2cfe5cda82e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d417a96a-edb7-4af2-8a95-2cfe5cda82e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:41.134822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:41.135752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 179/362] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:41.137415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417a96a-edb7-4af2-8a95-2cfe5cda82e8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:41.149905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:41.156559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053accc4-2a13-4d02-93e9-bfae1dd18e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:41.158244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053accc4-2a13-4d02-93e9-bfae1dd18e38 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:41.158522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053accc4-2a13-4d02-93e9-bfae1dd18e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:41.158794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053accc4-2a13-4d02-93e9-bfae1dd18e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:41.159635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053accc4-2a13-4d02-93e9-bfae1dd18e38 None admin] RBAC: Adding query filter params (group.id=e303cc6ee25c4dc8bd2f29042baab68d, role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=8ae24634160b4859b608dd29275497f2) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:41.160054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053accc4-2a13-4d02-93e9-bfae1dd18e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:41.162180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053accc4-2a13-4d02-93e9-bfae1dd18e38 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:41.169314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 184/363] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:40 2022] GET /identity/v3/role_assignments?group.id=e303cc6ee25c4dc8bd2f29042baab68d&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:41.173796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053accc4-2a13-4d02-93e9-bfae1dd18e38 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:41.286097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417a96a-edb7-4af2-8a95-2cfe5cda82e8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:41.330700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e0a63-c4bf-42cd-87e5-9a102330f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:41.331439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e0a63-c4bf-42cd-87e5-9a102330f2ce None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:41.331726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e0a63-c4bf-42cd-87e5-9a102330f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:41.332002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e0a63-c4bf-42cd-87e5-9a102330f2ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:41.333273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e0a63-c4bf-42cd-87e5-9a102330f2ce None admin] RBAC: Adding query filter params (group.id=e303cc6ee25c4dc8bd2f29042baab68d, role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=8ae24634160b4859b608dd29275497f2) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:41.333740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e0a63-c4bf-42cd-87e5-9a102330f2ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:41.335328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e0a63-c4bf-42cd-87e5-9a102330f2ce None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:41.335662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053accc4-2a13-4d02-93e9-bfae1dd18e38 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:41.342557 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 180/364] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:41 2022] GET /identity/v3/role_assignments?group.id=e303cc6ee25c4dc8bd2f29042baab68d&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:41.343401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e0a63-c4bf-42cd-87e5-9a102330f2ce None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:41.380435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1a898-333b-468e-9586-24d7d8c68995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:41.381611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1a898-333b-468e-9586-24d7d8c68995 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:41.382183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1a898-333b-468e-9586-24d7d8c68995 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:41.382469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1a898-333b-468e-9586-24d7d8c68995 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:41.383253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1a898-333b-468e-9586-24d7d8c68995 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:41.383563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1a898-333b-468e-9586-24d7d8c68995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:41.385031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1a898-333b-468e-9586-24d7d8c68995 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:41.396879 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 185/365] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:41.397961 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880b43f-26ad-4750-929d-70372396d0dd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:41.398254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880b43f-26ad-4750-929d-70372396d0dd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:41.398548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880b43f-26ad-4750-929d-70372396d0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:41.412559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:41.420550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0880b43f-26ad-4750-929d-70372396d0dd None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:41.421992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0880b43f-26ad-4750-929d-70372396d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:41.485701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e0a63-c4bf-42cd-87e5-9a102330f2ce None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:41.510229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:41.537406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 186/366] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:41.539002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880b43f-26ad-4750-929d-70372396d0dd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:41.539392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af0cd-d27e-47fc-8968-a5f081334931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:41.540137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af0cd-d27e-47fc-8968-a5f081334931 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:41.540493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af0cd-d27e-47fc-8968-a5f081334931 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:41.540860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af0cd-d27e-47fc-8968-a5f081334931 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:41.541830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af0cd-d27e-47fc-8968-a5f081334931 None admin] RBAC: Adding query filter params (group.id=e303cc6ee25c4dc8bd2f29042baab68d, role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=46bc7b47b92148ffb91da999460d824f) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:41.542208 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af0cd-d27e-47fc-8968-a5f081334931 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:41.543734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af0cd-d27e-47fc-8968-a5f081334931 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:41.557969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 181/367] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:41 2022] GET /identity/v3/role_assignments?group.id=e303cc6ee25c4dc8bd2f29042baab68d&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:41.558182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af0cd-d27e-47fc-8968-a5f081334931 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:41.674133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af0cd-d27e-47fc-8968-a5f081334931 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:41.689612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880b43f-26ad-4750-929d-70372396d0dd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:41.693310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:41.714260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1539ae7-74d9-414e-9343-d7dc3ff2e8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:41.715416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1539ae7-74d9-414e-9343-d7dc3ff2e8fd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:41.715416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1539ae7-74d9-414e-9343-d7dc3ff2e8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:41.715800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1539ae7-74d9-414e-9343-d7dc3ff2e8fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:41.732207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0643e-f434-4c2e-a273-ba3d6a6f4b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:41.733552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0643e-f434-4c2e-a273-ba3d6a6f4b17 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:41.734106 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0643e-f434-4c2e-a273-ba3d6a6f4b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:41.734561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0643e-f434-4c2e-a273-ba3d6a6f4b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:41.735516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0643e-f434-4c2e-a273-ba3d6a6f4b17 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:41.735965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0643e-f434-4c2e-a273-ba3d6a6f4b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:41.737649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0643e-f434-4c2e-a273-ba3d6a6f4b17 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:41.749276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 187/368] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:41.750534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0643e-f434-4c2e-a273-ba3d6a6f4b17 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:41.799334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1539ae7-74d9-414e-9343-d7dc3ff2e8fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:41.800633 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1539ae7-74d9-414e-9343-d7dc3ff2e8fd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:41.864771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:41.876661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0643e-f434-4c2e-a273-ba3d6a6f4b17 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:41.917687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1721a3-91b7-4565-be66-794cafba8474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:41.918642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1721a3-91b7-4565-be66-794cafba8474 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:41.919019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1721a3-91b7-4565-be66-794cafba8474 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:41.919547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1721a3-91b7-4565-be66-794cafba8474 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:41.930542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1721a3-91b7-4565-be66-794cafba8474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:41.931350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 182/369] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:41.932415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1539ae7-74d9-414e-9343-d7dc3ff2e8fd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3e1721a3-91b7-4565-be66-794cafba8474 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:41.938771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:41.939871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:41.940893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:41.942852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 188/370] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:22:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:41.943725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1721a3-91b7-4565-be66-794cafba8474 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:41.957012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:42.060235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1721a3-91b7-4565-be66-794cafba8474 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:42.064773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1539ae7-74d9-414e-9343-d7dc3ff2e8fd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:42.071139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:42.102976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678717b1-d736-4f75-a684-9759508ba097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:42.104021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678717b1-d736-4f75-a684-9759508ba097 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:42.104408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678717b1-d736-4f75-a684-9759508ba097 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:42.104821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678717b1-d736-4f75-a684-9759508ba097 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:42.106274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678717b1-d736-4f75-a684-9759508ba097 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:42.107676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678717b1-d736-4f75-a684-9759508ba097 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:42.110503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 189/371] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:41 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:42.110912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678717b1-d736-4f75-a684-9759508ba097 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:42.117703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09046706-adcd-4bf3-a8df-dbe96b73692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:42.118400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09046706-adcd-4bf3-a8df-dbe96b73692a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:42.120190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09046706-adcd-4bf3-a8df-dbe96b73692a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:42.120190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09046706-adcd-4bf3-a8df-dbe96b73692a None admin] PATH_INFO: `/v3/users/a26c236d252847eebf01b34bd3c3be39` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:42.121357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09046706-adcd-4bf3-a8df-dbe96b73692a None admin] RBAC: Authorizing `identity:get_user(user_id=a26c236d252847eebf01b34bd3c3be39)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:42.122510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09046706-adcd-4bf3-a8df-dbe96b73692a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:42.130387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 183/372] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:22:41 2022] GET /identity/v3/users/a26c236d252847eebf01b34bd3c3be39 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:42.130970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09046706-adcd-4bf3-a8df-dbe96b73692a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:42.274202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:42.294549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09046706-adcd-4bf3-a8df-dbe96b73692a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:42.317700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678717b1-d736-4f75-a684-9759508ba097 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:42.358581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5575000-f3e9-4f3c-b9d6-494f760b0cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:42.359452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575000-f3e9-4f3c-b9d6-494f760b0cbd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:42.359730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575000-f3e9-4f3c-b9d6-494f760b0cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:42.360010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575000-f3e9-4f3c-b9d6-494f760b0cbd None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:42.361272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5575000-f3e9-4f3c-b9d6-494f760b0cbd None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:42.363615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5575000-f3e9-4f3c-b9d6-494f760b0cbd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:42.368906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 184/373] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:42 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:42.370413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5575000-f3e9-4f3c-b9d6-494f760b0cbd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:42.385061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8537eb-5afd-473c-8cbf-0afa03c06a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:42.385749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8537eb-5afd-473c-8cbf-0afa03c06a27 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:42.387813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8537eb-5afd-473c-8cbf-0afa03c06a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:42.387813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8537eb-5afd-473c-8cbf-0afa03c06a27 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:42.387813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8537eb-5afd-473c-8cbf-0afa03c06a27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:42.387813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8537eb-5afd-473c-8cbf-0afa03c06a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:42.388706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8537eb-5afd-473c-8cbf-0afa03c06a27 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:42.403757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 190/374] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:42.403757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8537eb-5afd-473c-8cbf-0afa03c06a27 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:42.445011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:42.512226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8537eb-5afd-473c-8cbf-0afa03c06a27 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:42.548179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1dac0-358a-4052-a5d2-cb2e988741f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:42.549583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1dac0-358a-4052-a5d2-cb2e988741f3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:42.550069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1dac0-358a-4052-a5d2-cb2e988741f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:42.550529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1dac0-358a-4052-a5d2-cb2e988741f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:42.551543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1dac0-358a-4052-a5d2-cb2e988741f3 None admin] RBAC: Adding query filter params (group.id=e303cc6ee25c4dc8bd2f29042baab68d, role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=46bc7b47b92148ffb91da999460d824f) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:42.552023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1dac0-358a-4052-a5d2-cb2e988741f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:42.553089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5575000-f3e9-4f3c-b9d6-494f760b0cbd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:42.554150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1dac0-358a-4052-a5d2-cb2e988741f3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:42.560212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 191/375] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:42 2022] GET /identity/v3/role_assignments?group.id=e303cc6ee25c4dc8bd2f29042baab68d&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:42.561304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1dac0-358a-4052-a5d2-cb2e988741f3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:42.621612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c50bb6-a60a-47fe-80fa-d6dedd602ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:42.622427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c50bb6-a60a-47fe-80fa-d6dedd602ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:42.622785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c50bb6-a60a-47fe-80fa-d6dedd602ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:42.623140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c50bb6-a60a-47fe-80fa-d6dedd602ce5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:42.624149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c50bb6-a60a-47fe-80fa-d6dedd602ce5 None admin] RBAC: Adding query filter params (role.id=c0d100940f624679824541cd7101a203, user.id=a26c236d252847eebf01b34bd3c3be39, scope.system=all) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:42.624467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c50bb6-a60a-47fe-80fa-d6dedd602ce5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:42.626198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c50bb6-a60a-47fe-80fa-d6dedd602ce5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:42.648220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 185/376] 10.222.0.69 () {56 vars in 1308 bytes} [Fri Jun 24 10:22:42 2022] GET /identity/v3/role_assignments?user.id=a26c236d252847eebf01b34bd3c3be39&scope.system=all&role.id=c0d100940f624679824541cd7101a203 => generated 510 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:42.676628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1dac0-358a-4052-a5d2-cb2e988741f3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:42.678510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:42.720514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79727586-9756-4f34-a3bd-f54a07be5fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:42.721232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79727586-9756-4f34-a3bd-f54a07be5fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:42.721519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79727586-9756-4f34-a3bd-f54a07be5fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:42.721802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79727586-9756-4f34-a3bd-f54a07be5fc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:42.800085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79727586-9756-4f34-a3bd-f54a07be5fc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:42.801596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79727586-9756-4f34-a3bd-f54a07be5fc7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:42.912096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 192/377] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:42.917839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c50bb6-a60a-47fe-80fa-d6dedd602ce5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:42.934475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:43.032353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c50bb6-a60a-47fe-80fa-d6dedd602ce5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:43.068379 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425aad6-8a10-4f2c-bcee-585c9a774b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:43.069193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425aad6-8a10-4f2c-bcee-585c9a774b6c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:43.069474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425aad6-8a10-4f2c-bcee-585c9a774b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:43.069746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425aad6-8a10-4f2c-bcee-585c9a774b6c None admin] PATH_INFO: `/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:43.070978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425aad6-8a10-4f2c-bcee-585c9a774b6c None admin] RBAC: Authorizing `identity:get_project(project_id=0f9dd8794ad7494bbf369d7c8ce42c78)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:43.072426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425aad6-8a10-4f2c-bcee-585c9a774b6c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:43.074452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 186/378] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:42 2022] GET /identity/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:43.080613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79727586-9756-4f34-a3bd-f54a07be5fc7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:43.082833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:43.191938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79727586-9756-4f34-a3bd-f54a07be5fc7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:43.228366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4787-3acb-403d-9746-51553cfecee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:43.229265 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4787-3acb-403d-9746-51553cfecee1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:43.229547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4787-3acb-403d-9746-51553cfecee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:43.229823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4787-3acb-403d-9746-51553cfecee1 None admin] PATH_INFO: `/v3/groups/e8cc7b318e714e968fbb8416575c5a99` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:43.236187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c4787-3acb-403d-9746-51553cfecee1 None admin] RBAC: Authorizing `identity:get_group(group_id=e8cc7b318e714e968fbb8416575c5a99)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:43.237671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c4787-3acb-403d-9746-51553cfecee1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:43.239631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 193/379] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:43 2022] GET /identity/v3/groups/e8cc7b318e714e968fbb8416575c5a99 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:43.245597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425aad6-8a10-4f2c-bcee-585c9a774b6c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:43.331279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:43.352429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425aad6-8a10-4f2c-bcee-585c9a774b6c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:43.392000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1482106-35e2-4883-909a-af50340f8716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:43.392668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482106-35e2-4883-909a-af50340f8716 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:43.392982 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482106-35e2-4883-909a-af50340f8716 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:43.393262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482106-35e2-4883-909a-af50340f8716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:43.394106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1482106-35e2-4883-909a-af50340f8716 None admin] RBAC: Adding query filter params (group.id=e8cc7b318e714e968fbb8416575c5a99, role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=0f9dd8794ad7494bbf369d7c8ce42c78) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:43.394417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1482106-35e2-4883-909a-af50340f8716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:43.395849 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1482106-35e2-4883-909a-af50340f8716 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:43.401945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 187/380] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:43 2022] GET /identity/v3/role_assignments?group.id=e8cc7b318e714e968fbb8416575c5a99&scope.project.id=0f9dd8794ad7494bbf369d7c8ce42c78&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:44.027312 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03923cae-ebfc-49e7-81d7-8ae8a31f2936 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.027852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03923cae-ebfc-49e7-81d7-8ae8a31f2936 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.027852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03923cae-ebfc-49e7-81d7-8ae8a31f2936 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.029502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 194/381] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:44.036345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45de59-6656-4fe4-bced-cd64225c0e3a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.036804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45de59-6656-4fe4-bced-cd64225c0e3a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.037009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45de59-6656-4fe4-bced-cd64225c0e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.056254 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bf45de59-6656-4fe4-bced-cd64225c0e3a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:44.057487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bf45de59-6656-4fe4-bced-cd64225c0e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:44.139620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 188/382] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:44.174007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03923cae-ebfc-49e7-81d7-8ae8a31f2936 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:44.267425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:44.314517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03923cae-ebfc-49e7-81d7-8ae8a31f2936 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:44.360381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5568ae5-9de8-4200-8fd4-6534722cf5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:44.361227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5568ae5-9de8-4200-8fd4-6534722cf5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.361509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5568ae5-9de8-4200-8fd4-6534722cf5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.361787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5568ae5-9de8-4200-8fd4-6534722cf5e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.374897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5568ae5-9de8-4200-8fd4-6534722cf5e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d5568ae5-9de8-4200-8fd4-6534722cf5e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:44.382756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:44.383632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:44.384867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 195/383] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:44.390345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45de59-6656-4fe4-bced-cd64225c0e3a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:44.496421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:44.529889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602eab-83e9-4cd4-9227-746217cf4ecc None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.530239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602eab-83e9-4cd4-9227-746217cf4ecc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.530448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602eab-83e9-4cd4-9227-746217cf4ecc None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.531832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 196/384] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:44.536877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899aa9b2-1d50-42c2-94d5-68783599ad1b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.537166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899aa9b2-1d50-42c2-94d5-68783599ad1b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.537447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899aa9b2-1d50-42c2-94d5-68783599ad1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.552387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45de59-6656-4fe4-bced-cd64225c0e3a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:44.567136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-899aa9b2-1d50-42c2-94d5-68783599ad1b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:44.568261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-899aa9b2-1d50-42c2-94d5-68783599ad1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:44.617022 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc6e64-3deb-4317-96fe-1d5396220b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:44.617709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc6e64-3deb-4317-96fe-1d5396220b1a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.617991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc6e64-3deb-4317-96fe-1d5396220b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.618266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc6e64-3deb-4317-96fe-1d5396220b1a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.619031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc6e64-3deb-4317-96fe-1d5396220b1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:44.619339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc6e64-3deb-4317-96fe-1d5396220b1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:44.620725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc6e64-3deb-4317-96fe-1d5396220b1a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:44.632542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 189/385] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:44.635124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e02ac-b905-4cb0-813a-b1280b4607e0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.635414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e02ac-b905-4cb0-813a-b1280b4607e0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.635695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e02ac-b905-4cb0-813a-b1280b4607e0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.637206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 190/386] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:44.637908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e02ac-b905-4cb0-813a-b1280b4607e0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:44.671254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 197/387] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:44.672538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b0313-d317-4a76-b467-4e95918224d3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.677602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b0313-d317-4a76-b467-4e95918224d3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.677905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b0313-d317-4a76-b467-4e95918224d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.706999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4c4b0313-d317-4a76-b467-4e95918224d3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:44.708183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4c4b0313-d317-4a76-b467-4e95918224d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:44.760830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:44.769103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e02ac-b905-4cb0-813a-b1280b4607e0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:44.780619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 198/388] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:44.781807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b0313-d317-4a76-b467-4e95918224d3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:44.804883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c6d37-656d-46c5-9483-a13f09f5da4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:44.805886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c6d37-656d-46c5-9483-a13f09f5da4a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.806172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c6d37-656d-46c5-9483-a13f09f5da4a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.806436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c6d37-656d-46c5-9483-a13f09f5da4a None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.807608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c6d37-656d-46c5-9483-a13f09f5da4a None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:44.809140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c6d37-656d-46c5-9483-a13f09f5da4a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:44.810774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 191/389] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:44 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:44.816342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5462b-2ad6-4184-8aac-539820893198 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.816634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5462b-2ad6-4184-8aac-539820893198 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.816959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5462b-2ad6-4184-8aac-539820893198 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.818346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 192/390] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:44.818851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5462b-2ad6-4184-8aac-539820893198 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:44.823114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:44.896773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b0313-d317-4a76-b467-4e95918224d3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:44.929132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5462b-2ad6-4184-8aac-539820893198 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:44.935505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109b04b-47d5-49e3-940a-8313086554ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:44.936370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109b04b-47d5-49e3-940a-8313086554ad None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.936654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109b04b-47d5-49e3-940a-8313086554ad None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.936965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109b04b-47d5-49e3-940a-8313086554ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.947258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109b04b-47d5-49e3-940a-8313086554ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-1109b04b-47d5-49e3-940a-8313086554ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:44.955286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:44.956239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:44.957110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 199/391] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:44.957787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109b04b-47d5-49e3-940a-8313086554ad None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:44.969990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34d4d1-27ac-40a0-8e58-562e0f9a57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:44.970810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34d4d1-27ac-40a0-8e58-562e0f9a57fe None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.971264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34d4d1-27ac-40a0-8e58-562e0f9a57fe None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.971374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34d4d1-27ac-40a0-8e58-562e0f9a57fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:44.983175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34d4d1-27ac-40a0-8e58-562e0f9a57fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2f34d4d1-27ac-40a0-8e58-562e0f9a57fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:44.991227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:44.992101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:44.992934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 193/392] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:44.993609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50680-71a8-4025-a6fd-eb8f751209e9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:44.994287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50680-71a8-4025-a6fd-eb8f751209e9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:44.994287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50680-71a8-4025-a6fd-eb8f751209e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.011147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3df50680-71a8-4025-a6fd-eb8f751209e9 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:45.012331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3df50680-71a8-4025-a6fd-eb8f751209e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:45.013596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:45.068964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109b04b-47d5-49e3-940a-8313086554ad None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.076131 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:45.087156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 194/393] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:45.087799 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50680-71a8-4025-a6fd-eb8f751209e9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.112821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5a62f-9c6d-4e41-87f9-7383fafe575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.113728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5a62f-9c6d-4e41-87f9-7383fafe575a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.114012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5a62f-9c6d-4e41-87f9-7383fafe575a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.114291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5a62f-9c6d-4e41-87f9-7383fafe575a None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.115491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5a62f-9c6d-4e41-87f9-7383fafe575a None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:45.116979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5a62f-9c6d-4e41-87f9-7383fafe575a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:45.118822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 200/394] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:44 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:45.120789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5a62f-9c6d-4e41-87f9-7383fafe575a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.202443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50680-71a8-4025-a6fd-eb8f751209e9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.228946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5a62f-9c6d-4e41-87f9-7383fafe575a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.240067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42164ed4-86b6-413b-a7a6-e3fd1ae18e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.240752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42164ed4-86b6-413b-a7a6-e3fd1ae18e5a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.241037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42164ed4-86b6-413b-a7a6-e3fd1ae18e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.241316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42164ed4-86b6-413b-a7a6-e3fd1ae18e5a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.242066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42164ed4-86b6-413b-a7a6-e3fd1ae18e5a None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:45.242367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42164ed4-86b6-413b-a7a6-e3fd1ae18e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:45.243731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42164ed4-86b6-413b-a7a6-e3fd1ae18e5a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:45.245612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:45.252576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 195/395] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:45.253494 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42164ed4-86b6-413b-a7a6-e3fd1ae18e5a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.268024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe2aea-047e-46e5-b0d1-997d7e65c9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.268693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe2aea-047e-46e5-b0d1-997d7e65c9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.269014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe2aea-047e-46e5-b0d1-997d7e65c9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.269348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe2aea-047e-46e5-b0d1-997d7e65c9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.270104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe2aea-047e-46e5-b0d1-997d7e65c9c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:45.270463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe2aea-047e-46e5-b0d1-997d7e65c9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:45.271873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe2aea-047e-46e5-b0d1-997d7e65c9c3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:45.280271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 201/396] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:45.281229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe2aea-047e-46e5-b0d1-997d7e65c9c3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.319119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:45.364236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42164ed4-86b6-413b-a7a6-e3fd1ae18e5a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.386070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe2aea-047e-46e5-b0d1-997d7e65c9c3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.398228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f38e1-7094-4744-97a6-b9922289b8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.399052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f38e1-7094-4744-97a6-b9922289b8cc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.399332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f38e1-7094-4744-97a6-b9922289b8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.399604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f38e1-7094-4744-97a6-b9922289b8cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.410216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f38e1-7094-4744-97a6-b9922289b8cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-853f38e1-7094-4744-97a6-b9922289b8cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:45.418258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:45.419118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:45.419883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 196/397] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:45.420890 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f38e1-7094-4744-97a6-b9922289b8cc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.430958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40a181-e3e4-48ce-9752-494fb6404c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.432159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40a181-e3e4-48ce-9752-494fb6404c61 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.432647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40a181-e3e4-48ce-9752-494fb6404c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.433045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40a181-e3e4-48ce-9752-494fb6404c61 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2/groups/e303cc6ee25c4dc8bd2f29042baab68d/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.435702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40a181-e3e4-48ce-9752-494fb6404c61 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ae24634160b4859b608dd29275497f2, group_id=e303cc6ee25c4dc8bd2f29042baab68d, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:45.437403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40a181-e3e4-48ce-9752-494fb6404c61 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:45.453880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 202/398] 10.222.0.69 () {58 vars in 1442 bytes} [Fri Jun 24 10:22:45 2022] PUT /identity/v3/projects/8ae24634160b4859b608dd29275497f2/groups/e303cc6ee25c4dc8bd2f29042baab68d/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:45.454909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40a181-e3e4-48ce-9752-494fb6404c61 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.524874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:45.556604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f38e1-7094-4744-97a6-b9922289b8cc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.576998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:45.603367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42b8fe-19e9-494d-a94e-de1e5aea5100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.604438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42b8fe-19e9-494d-a94e-de1e5aea5100 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.604728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42b8fe-19e9-494d-a94e-de1e5aea5100 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.605051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42b8fe-19e9-494d-a94e-de1e5aea5100 None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.606748 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42b8fe-19e9-494d-a94e-de1e5aea5100 None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:45.608455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42b8fe-19e9-494d-a94e-de1e5aea5100 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:45.612888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 197/399] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:45 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:45.614057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42b8fe-19e9-494d-a94e-de1e5aea5100 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.629457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40a181-e3e4-48ce-9752-494fb6404c61 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.678765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f253d67-4b64-461e-9b3f-1ad88bc69902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.679629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f253d67-4b64-461e-9b3f-1ad88bc69902 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.679910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f253d67-4b64-461e-9b3f-1ad88bc69902 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.680295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f253d67-4b64-461e-9b3f-1ad88bc69902 None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.681519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f253d67-4b64-461e-9b3f-1ad88bc69902 None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:45.683719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f253d67-4b64-461e-9b3f-1ad88bc69902 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:45.688820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 203/400] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:45 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:45.689941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f253d67-4b64-461e-9b3f-1ad88bc69902 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.742977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42b8fe-19e9-494d-a94e-de1e5aea5100 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.779227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd2033-ab4c-457d-8712-14da0b67813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.779988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd2033-ab4c-457d-8712-14da0b67813e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.780358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd2033-ab4c-457d-8712-14da0b67813e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.780660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd2033-ab4c-457d-8712-14da0b67813e None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.781591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd2033-ab4c-457d-8712-14da0b67813e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:45.781897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd2033-ab4c-457d-8712-14da0b67813e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:45.783265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd2033-ab4c-457d-8712-14da0b67813e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:45.796242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 198/401] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:45.796242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd2033-ab4c-457d-8712-14da0b67813e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.805069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f253d67-4b64-461e-9b3f-1ad88bc69902 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.807794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:45.842077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:45.845445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c30b17-e970-4973-b458-ccae8f7b9926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.846639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c30b17-e970-4973-b458-ccae8f7b9926 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.847130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c30b17-e970-4973-b458-ccae8f7b9926 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.847598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c30b17-e970-4973-b458-ccae8f7b9926 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.849281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c30b17-e970-4973-b458-ccae8f7b9926 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:45.851020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c30b17-e970-4973-b458-ccae8f7b9926 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:45.853081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 204/402] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:45 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:45.854064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c30b17-e970-4973-b458-ccae8f7b9926 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.899096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd2033-ab4c-457d-8712-14da0b67813e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:45.933785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8b9db-d943-457a-84f5-a1bc8a61b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:45.935079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8b9db-d943-457a-84f5-a1bc8a61b85d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.935398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8b9db-d943-457a-84f5-a1bc8a61b85d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.935673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8b9db-d943-457a-84f5-a1bc8a61b85d None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.937369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8b9db-d943-457a-84f5-a1bc8a61b85d None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:45.939229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8b9db-d943-457a-84f5-a1bc8a61b85d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:45.945595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 199/403] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:45 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:45.946995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076dfd9-a1cb-46d1-a21c-20141b56759b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:45.947281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076dfd9-a1cb-46d1-a21c-20141b56759b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:45.947554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076dfd9-a1cb-46d1-a21c-20141b56759b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:45.949673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 200/404] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:45.949673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076dfd9-a1cb-46d1-a21c-20141b56759b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:45.962912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c30b17-e970-4973-b458-ccae8f7b9926 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:46.002151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e1027-3822-45e2-acf3-e4c97c84b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:46.003012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e1027-3822-45e2-acf3-e4c97c84b39e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:46.003291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e1027-3822-45e2-acf3-e4c97c84b39e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:46.003564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e1027-3822-45e2-acf3-e4c97c84b39e None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:46.004727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e1027-3822-45e2-acf3-e4c97c84b39e None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:46.006188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e1027-3822-45e2-acf3-e4c97c84b39e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:46.007973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 205/405] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:45 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:46.008598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e1027-3822-45e2-acf3-e4c97c84b39e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:46.051189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:46.065793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076dfd9-a1cb-46d1-a21c-20141b56759b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:46.079316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:46.100115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d337b1-a27a-402e-9f18-e7ddab3c8c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:46.100938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d337b1-a27a-402e-9f18-e7ddab3c8c0c None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:46.101224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d337b1-a27a-402e-9f18-e7ddab3c8c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:46.101506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d337b1-a27a-402e-9f18-e7ddab3c8c0c None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f/groups/e303cc6ee25c4dc8bd2f29042baab68d/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:46.103460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d337b1-a27a-402e-9f18-e7ddab3c8c0c None admin] RBAC: Authorizing `identity:create_grant(project_id=46bc7b47b92148ffb91da999460d824f, group_id=e303cc6ee25c4dc8bd2f29042baab68d, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:46.104944 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d337b1-a27a-402e-9f18-e7ddab3c8c0c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:46.116012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e1027-3822-45e2-acf3-e4c97c84b39e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:46.118999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 201/406] 10.222.0.69 () {58 vars in 1442 bytes} [Fri Jun 24 10:22:45 2022] PUT /identity/v3/projects/46bc7b47b92148ffb91da999460d824f/groups/e303cc6ee25c4dc8bd2f29042baab68d/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:46.120162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813b5d9-3267-42ef-92c7-62fe41960e69 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:46.120445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813b5d9-3267-42ef-92c7-62fe41960e69 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:46.120718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813b5d9-3267-42ef-92c7-62fe41960e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:46.139445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e813b5d9-3267-42ef-92c7-62fe41960e69 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:46.140839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e813b5d9-3267-42ef-92c7-62fe41960e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:46.165712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ded36b-18e2-4417-b4cc-c856a9c30791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:46.167258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ded36b-18e2-4417-b4cc-c856a9c30791 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:46.167548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ded36b-18e2-4417-b4cc-c856a9c30791 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:46.168114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ded36b-18e2-4417-b4cc-c856a9c30791 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2/groups/e303cc6ee25c4dc8bd2f29042baab68d/roles/bde14daa0b464f6f9622e4d84e28e7d1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:46.171358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ded36b-18e2-4417-b4cc-c856a9c30791 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ae24634160b4859b608dd29275497f2, group_id=e303cc6ee25c4dc8bd2f29042baab68d, role_id=bde14daa0b464f6f9622e4d84e28e7d1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:46.173439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ded36b-18e2-4417-b4cc-c856a9c30791 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:46.193015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 206/407] 10.222.0.69 () {58 vars in 1442 bytes} [Fri Jun 24 10:22:46 2022] PUT /identity/v3/projects/8ae24634160b4859b608dd29275497f2/groups/e303cc6ee25c4dc8bd2f29042baab68d/roles/bde14daa0b464f6f9622e4d84e28e7d1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:46.194017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ded36b-18e2-4417-b4cc-c856a9c30791 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:46.226022 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 202/408] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:46.250158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813b5d9-3267-42ef-92c7-62fe41960e69 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:46.329404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:46.332623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ded36b-18e2-4417-b4cc-c856a9c30791 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:46.377760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b013dcd-49a1-444c-a8c2-e1271f3cd35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:46.378203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813b5d9-3267-42ef-92c7-62fe41960e69 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:46.378567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b013dcd-49a1-444c-a8c2-e1271f3cd35b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:46.378851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b013dcd-49a1-444c-a8c2-e1271f3cd35b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:46.379130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b013dcd-49a1-444c-a8c2-e1271f3cd35b None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:46.380420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b013dcd-49a1-444c-a8c2-e1271f3cd35b None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:46.382149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b013dcd-49a1-444c-a8c2-e1271f3cd35b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:46.382438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:46.388763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 207/409] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:46 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:46.394067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b013dcd-49a1-444c-a8c2-e1271f3cd35b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:46.453661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481042c-5c8a-4bfb-99a8-36abdbeef463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:46.454485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481042c-5c8a-4bfb-99a8-36abdbeef463 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:46.455087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481042c-5c8a-4bfb-99a8-36abdbeef463 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:46.455361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481042c-5c8a-4bfb-99a8-36abdbeef463 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:46.474270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481042c-5c8a-4bfb-99a8-36abdbeef463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c481042c-5c8a-4bfb-99a8-36abdbeef463 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:46.483066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:46.483926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:46.484722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:46.487373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 203/410] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:46.493596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481042c-5c8a-4bfb-99a8-36abdbeef463 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:46.531921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b013dcd-49a1-444c-a8c2-e1271f3cd35b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:46.605765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded477e-ea7c-43aa-bb6a-3e3c8816bc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:46.607197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded477e-ea7c-43aa-bb6a-3e3c8816bc84 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:46.607754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded477e-ea7c-43aa-bb6a-3e3c8816bc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:46.608561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded477e-ea7c-43aa-bb6a-3e3c8816bc84 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f/groups/e303cc6ee25c4dc8bd2f29042baab68d/roles/bde14daa0b464f6f9622e4d84e28e7d1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:46.616141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded477e-ea7c-43aa-bb6a-3e3c8816bc84 None admin] RBAC: Authorizing `identity:create_grant(project_id=46bc7b47b92148ffb91da999460d824f, group_id=e303cc6ee25c4dc8bd2f29042baab68d, role_id=bde14daa0b464f6f9622e4d84e28e7d1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:46.618269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded477e-ea7c-43aa-bb6a-3e3c8816bc84 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:46.628102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481042c-5c8a-4bfb-99a8-36abdbeef463 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:46.651126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 208/411] 10.222.0.69 () {58 vars in 1442 bytes} [Fri Jun 24 10:22:46 2022] PUT /identity/v3/projects/46bc7b47b92148ffb91da999460d824f/groups/e303cc6ee25c4dc8bd2f29042baab68d/roles/bde14daa0b464f6f9622e4d84e28e7d1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:22:46.681787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bc891-74a8-44d8-ba57-1362074eedd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:46.682555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bc891-74a8-44d8-ba57-1362074eedd4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:46.682851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bc891-74a8-44d8-ba57-1362074eedd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:46.683133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bc891-74a8-44d8-ba57-1362074eedd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:46.683952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bc891-74a8-44d8-ba57-1362074eedd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:46.684262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bc891-74a8-44d8-ba57-1362074eedd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:46.685719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bc891-74a8-44d8-ba57-1362074eedd4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:46.696038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 204/412] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:46.701448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded477e-ea7c-43aa-bb6a-3e3c8816bc84 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:46.756353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:46.859891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded477e-ea7c-43aa-bb6a-3e3c8816bc84 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:46.898665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413bf59-569a-480a-b644-6301b2e3f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:46.899633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413bf59-569a-480a-b644-6301b2e3f936 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:46.900017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413bf59-569a-480a-b644-6301b2e3f936 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:46.900389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413bf59-569a-480a-b644-6301b2e3f936 None admin] PATH_INFO: `/v3/groups/e8cc7b318e714e968fbb8416575c5a99` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:46.901723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413bf59-569a-480a-b644-6301b2e3f936 None admin] RBAC: Authorizing `identity:get_group(group_id=e8cc7b318e714e968fbb8416575c5a99)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:46.903280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413bf59-569a-480a-b644-6301b2e3f936 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:46.906019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 209/413] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:46 2022] GET /identity/v3/groups/e8cc7b318e714e968fbb8416575c5a99 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:46.911186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bc891-74a8-44d8-ba57-1362074eedd4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:46.962551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:47.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bc891-74a8-44d8-ba57-1362074eedd4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:47.117998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63b532-d99a-431d-84c5-de411ea407cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:47.118880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63b532-d99a-431d-84c5-de411ea407cd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:47.119164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63b532-d99a-431d-84c5-de411ea407cd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:47.119439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63b532-d99a-431d-84c5-de411ea407cd None admin] PATH_INFO: `/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:47.120638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63b532-d99a-431d-84c5-de411ea407cd None admin] RBAC: Authorizing `identity:get_project(project_id=0f9dd8794ad7494bbf369d7c8ce42c78)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:47.122105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63b532-d99a-431d-84c5-de411ea407cd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:47.124211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 205/414] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:46 2022] GET /identity/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78 => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:47.130215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413bf59-569a-480a-b644-6301b2e3f936 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:47.267151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:47.273364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413bf59-569a-480a-b644-6301b2e3f936 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:47.319251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d2f8f-4505-4e27-a4fd-9528b3b4d646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:47.320147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d2f8f-4505-4e27-a4fd-9528b3b4d646 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:47.320425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d2f8f-4505-4e27-a4fd-9528b3b4d646 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:47.320698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d2f8f-4505-4e27-a4fd-9528b3b4d646 None admin] PATH_INFO: `/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78/groups/e8cc7b318e714e968fbb8416575c5a99/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:47.322674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d2f8f-4505-4e27-a4fd-9528b3b4d646 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f9dd8794ad7494bbf369d7c8ce42c78, group_id=e8cc7b318e714e968fbb8416575c5a99, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:47.324118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d2f8f-4505-4e27-a4fd-9528b3b4d646 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:47.348528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 210/415] 10.222.0.69 () {58 vars in 1442 bytes} [Fri Jun 24 10:22:47 2022] PUT /identity/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78/groups/e8cc7b318e714e968fbb8416575c5a99/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:47.651122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63497d69-b5e6-4884-9ecb-45a941f2fd96 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:47.651312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63497d69-b5e6-4884-9ecb-45a941f2fd96 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:47.651580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63497d69-b5e6-4884-9ecb-45a941f2fd96 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:47.653368 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 206/416] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:47.659425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a854b-c8aa-43e0-b56e-575ae9dadaeb None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:47.659728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a854b-c8aa-43e0-b56e-575ae9dadaeb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:47.659989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a854b-c8aa-43e0-b56e-575ae9dadaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:47.679969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5d4a854b-c8aa-43e0-b56e-575ae9dadaeb None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:47.681294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5d4a854b-c8aa-43e0-b56e-575ae9dadaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:47.756846 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 211/417] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:47.776122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63497d69-b5e6-4884-9ecb-45a941f2fd96 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:47.897689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:47.905860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63497d69-b5e6-4884-9ecb-45a941f2fd96 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:47.953684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcb7f-1859-4168-a4c5-ebf0f61f1f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:47.954490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcb7f-1859-4168-a4c5-ebf0f61f1f7d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:47.955098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcb7f-1859-4168-a4c5-ebf0f61f1f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:47.955371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcb7f-1859-4168-a4c5-ebf0f61f1f7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:47.969374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bcb7f-1859-4168-a4c5-ebf0f61f1f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e00bcb7f-1859-4168-a4c5-ebf0f61f1f7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:47.977060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:47.977938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:47.978702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 207/418] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:47.984195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a854b-c8aa-43e0-b56e-575ae9dadaeb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:48.099856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:48.123734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a854b-c8aa-43e0-b56e-575ae9dadaeb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:48.170156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf9f35-b099-41fa-b56a-b6ab411a9e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:48.170927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf9f35-b099-41fa-b56a-b6ab411a9e5c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:48.171287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf9f35-b099-41fa-b56a-b6ab411a9e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:48.171623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf9f35-b099-41fa-b56a-b6ab411a9e5c None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:48.172394 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf9f35-b099-41fa-b56a-b6ab411a9e5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:48.172764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf9f35-b099-41fa-b56a-b6ab411a9e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:48.174155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf9f35-b099-41fa-b56a-b6ab411a9e5c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:48.185525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 212/419] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:48.191616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcb7f-1859-4168-a4c5-ebf0f61f1f7d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:48.309949 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcb7f-1859-4168-a4c5-ebf0f61f1f7d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:48.345270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe2e72-e34b-4f81-bb03-6c9f3e0d23bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:48.345927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe2e72-e34b-4f81-bb03-6c9f3e0d23bd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:48.346209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe2e72-e34b-4f81-bb03-6c9f3e0d23bd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:48.346485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe2e72-e34b-4f81-bb03-6c9f3e0d23bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:48.371956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:48.417735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe2e72-e34b-4f81-bb03-6c9f3e0d23bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:48.419214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe2e72-e34b-4f81-bb03-6c9f3e0d23bd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:48.521956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 208/420] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:48.527288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf9f35-b099-41fa-b56a-b6ab411a9e5c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:48.678394 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf9f35-b099-41fa-b56a-b6ab411a9e5c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:48.720417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27caf927-e2d3-4426-a486-81299b1c8f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:48.721247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caf927-e2d3-4426-a486-81299b1c8f7a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:48.721530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caf927-e2d3-4426-a486-81299b1c8f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:48.721807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caf927-e2d3-4426-a486-81299b1c8f7a None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:48.723022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27caf927-e2d3-4426-a486-81299b1c8f7a None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:48.724469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27caf927-e2d3-4426-a486-81299b1c8f7a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:48.726372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 213/421] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:48 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:48.731755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe2e72-e34b-4f81-bb03-6c9f3e0d23bd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:48.762948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625dbe7-c651-44f1-b5ba-f617eb4da6e0 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:48.763254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625dbe7-c651-44f1-b5ba-f617eb4da6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:48.763508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625dbe7-c651-44f1-b5ba-f617eb4da6e0 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:48.765584 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 214/422] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:48.774267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e993d-c22a-4861-acfc-49afc02b6248 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:48.774557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e993d-c22a-4861-acfc-49afc02b6248 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:48.774841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e993d-c22a-4861-acfc-49afc02b6248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:48.793871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-651e993d-c22a-4861-acfc-49afc02b6248 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:48.795071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-651e993d-c22a-4861-acfc-49afc02b6248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:48.819322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:48.839280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:48.904174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe2e72-e34b-4f81-bb03-6c9f3e0d23bd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:48.905596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 215/423] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:48.907391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84694c84-a8ca-4782-a3b7-12ea8d1913e0 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:48.907910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84694c84-a8ca-4782-a3b7-12ea8d1913e0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:48.908381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84694c84-a8ca-4782-a3b7-12ea8d1913e0 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:48.910102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 216/424] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:48.915623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dda83-311d-447a-8cce-79ce4224f4b9 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:48.916164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dda83-311d-447a-8cce-79ce4224f4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:48.916645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dda83-311d-447a-8cce-79ce4224f4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:48.934088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4c4dda83-311d-447a-8cce-79ce4224f4b9 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:48.935644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4c4dda83-311d-447a-8cce-79ce4224f4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:48.942358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f76768-2794-4019-9d64-ac2b9a7b3793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:48.943222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f76768-2794-4019-9d64-ac2b9a7b3793 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:48.943501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f76768-2794-4019-9d64-ac2b9a7b3793 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:48.943782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f76768-2794-4019-9d64-ac2b9a7b3793 None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:48.944984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f76768-2794-4019-9d64-ac2b9a7b3793 None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:48.946856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f76768-2794-4019-9d64-ac2b9a7b3793 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:48.949432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 209/425] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:48 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:48.950215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f76768-2794-4019-9d64-ac2b9a7b3793 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:49.010426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 217/426] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:49.011247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dda83-311d-447a-8cce-79ce4224f4b9 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:49.056031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f76768-2794-4019-9d64-ac2b9a7b3793 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:49.085173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:49.088443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:49.098792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52f5b0-ea1e-4f07-a4e8-9f368d74cc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:49.100214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52f5b0-ea1e-4f07-a4e8-9f368d74cc75 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:49.100651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52f5b0-ea1e-4f07-a4e8-9f368d74cc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:49.101099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52f5b0-ea1e-4f07-a4e8-9f368d74cc75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:49.118153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52f5b0-ea1e-4f07-a4e8-9f368d74cc75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:49.122849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dda83-311d-447a-8cce-79ce4224f4b9 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-eb52f5b0-ea1e-4f07-a4e8-9f368d74cc75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:49.126551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:49.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:22:49.128200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:49.132562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 210/427] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:49.133474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52f5b0-ea1e-4f07-a4e8-9f368d74cc75 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:49.171324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f81fd-e55f-4f80-b0fa-0d41ac7fc807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:49.172693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f81fd-e55f-4f80-b0fa-0d41ac7fc807 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:49.173205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f81fd-e55f-4f80-b0fa-0d41ac7fc807 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:49.173671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f81fd-e55f-4f80-b0fa-0d41ac7fc807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:49.175235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f81fd-e55f-4f80-b0fa-0d41ac7fc807 None admin] RBAC: Adding query filter params (group.id=e303cc6ee25c4dc8bd2f29042baab68d, role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=8ae24634160b4859b608dd29275497f2) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:49.175861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f81fd-e55f-4f80-b0fa-0d41ac7fc807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:49.178714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f81fd-e55f-4f80-b0fa-0d41ac7fc807 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:49.187970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 218/428] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:49 2022] GET /identity/v3/role_assignments?group.id=e303cc6ee25c4dc8bd2f29042baab68d&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:49.189080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f81fd-e55f-4f80-b0fa-0d41ac7fc807 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:49.264606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52f5b0-ea1e-4f07-a4e8-9f368d74cc75 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:49.302490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ec171-d33f-443b-9beb-5848bb4500d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:49.303576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ec171-d33f-443b-9beb-5848bb4500d2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:49.303924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ec171-d33f-443b-9beb-5848bb4500d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:49.304267 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ec171-d33f-443b-9beb-5848bb4500d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:49.316322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ec171-d33f-443b-9beb-5848bb4500d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-fb5ec171-d33f-443b-9beb-5848bb4500d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:49.325125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:49.326120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:49.326931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 211/429] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:49.328855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac37fa-fb56-4a6d-a56b-704b9e25bd03 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:49.329312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac37fa-fb56-4a6d-a56b-704b9e25bd03 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:49.329572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac37fa-fb56-4a6d-a56b-704b9e25bd03 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:49.331640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 212/430] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:49.332264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f81fd-e55f-4f80-b0fa-0d41ac7fc807 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:49.332747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac37fa-fb56-4a6d-a56b-704b9e25bd03 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:49.379919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ff9f8-3663-401b-a11d-c90964dd9194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:49.380957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ff9f8-3663-401b-a11d-c90964dd9194 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:49.381381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ff9f8-3663-401b-a11d-c90964dd9194 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:49.381662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ff9f8-3663-401b-a11d-c90964dd9194 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:49.382426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ff9f8-3663-401b-a11d-c90964dd9194 None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:49.382742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ff9f8-3663-401b-a11d-c90964dd9194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:49.384125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ff9f8-3663-401b-a11d-c90964dd9194 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:49.386161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:49.395696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 219/431] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:22:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:49.397068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f25cf-9d11-4061-af27-37a4b9c91b5e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:49.397358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f25cf-9d11-4061-af27-37a4b9c91b5e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:49.397637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f25cf-9d11-4061-af27-37a4b9c91b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:49.404279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:49.420347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-916f25cf-9d11-4061-af27-37a4b9c91b5e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:49.421585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-916f25cf-9d11-4061-af27-37a4b9c91b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:49.466179 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac37fa-fb56-4a6d-a56b-704b9e25bd03 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:49.516203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49bda0-291e-4e41-860f-466b60cdcc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:49.517005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 220/432] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:49.517125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49bda0-291e-4e41-860f-466b60cdcc54 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:49.517125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49bda0-291e-4e41-860f-466b60cdcc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:49.517376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49bda0-291e-4e41-860f-466b60cdcc54 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:49.517642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f25cf-9d11-4061-af27-37a4b9c91b5e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:49.518923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49bda0-291e-4e41-860f-466b60cdcc54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:49.518923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49bda0-291e-4e41-860f-466b60cdcc54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:49.520226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49bda0-291e-4e41-860f-466b60cdcc54 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:49.532639 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 213/433] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:49.536869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49bda0-291e-4e41-860f-466b60cdcc54 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:49.646959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f25cf-9d11-4061-af27-37a4b9c91b5e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:49.672430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49bda0-291e-4e41-860f-466b60cdcc54 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:49.676173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:49.689372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:49.695860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37159f50-ac6e-47fb-b629-3d42e209ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:49.696522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37159f50-ac6e-47fb-b629-3d42e209ac19 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:49.696894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37159f50-ac6e-47fb-b629-3d42e209ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:49.697621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37159f50-ac6e-47fb-b629-3d42e209ac19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:49.713343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd89c1-d585-4960-952a-cdde2ee5165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:49.714317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd89c1-d585-4960-952a-cdde2ee5165e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:49.714665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd89c1-d585-4960-952a-cdde2ee5165e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:49.715023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd89c1-d585-4960-952a-cdde2ee5165e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:49.770786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37159f50-ac6e-47fb-b629-3d42e209ac19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:49.772561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37159f50-ac6e-47fb-b629-3d42e209ac19 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:49.790621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd89c1-d585-4960-952a-cdde2ee5165e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:49.792075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd89c1-d585-4960-952a-cdde2ee5165e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:49.873122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 221/434] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:49.874080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37159f50-ac6e-47fb-b629-3d42e209ac19 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:49.901783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 214/435] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:49.902665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd89c1-d585-4960-952a-cdde2ee5165e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:49.908399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:49.910528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:49.979699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37159f50-ac6e-47fb-b629-3d42e209ac19 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.013974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd89c1-d585-4960-952a-cdde2ee5165e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.014443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a01e5-74f8-4004-8501-e375ce35d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.015246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a01e5-74f8-4004-8501-e375ce35d384 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.015500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a01e5-74f8-4004-8501-e375ce35d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.015770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a01e5-74f8-4004-8501-e375ce35d384 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.026437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a01e5-74f8-4004-8501-e375ce35d384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-689a01e5-74f8-4004-8501-e375ce35d384 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:50.034718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:50.035572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:50.036338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 222/436] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:50.037021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a01e5-74f8-4004-8501-e375ce35d384 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.053275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d134a75-54ff-4795-bd45-dcbc8f68a9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.054147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d134a75-54ff-4795-bd45-dcbc8f68a9fb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.054425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d134a75-54ff-4795-bd45-dcbc8f68a9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.054701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d134a75-54ff-4795-bd45-dcbc8f68a9fb None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.055878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d134a75-54ff-4795-bd45-dcbc8f68a9fb None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.057358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d134a75-54ff-4795-bd45-dcbc8f68a9fb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.059354 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 215/437] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:49 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:50.060495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811095b0-4f94-4821-a7a5-d1656dad71cf None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.060804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811095b0-4f94-4821-a7a5-d1656dad71cf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.061283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811095b0-4f94-4821-a7a5-d1656dad71cf None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.064869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 216/438] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:50.065721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811095b0-4f94-4821-a7a5-d1656dad71cf None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.125800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:50.134636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a01e5-74f8-4004-8501-e375ce35d384 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.135502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:50.167248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811095b0-4f94-4821-a7a5-d1656dad71cf None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.168178 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082d139-327f-45a7-a711-b8b520a00289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.169070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d139-327f-45a7-a711-b8b520a00289 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.169350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d139-327f-45a7-a711-b8b520a00289 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.169627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d139-327f-45a7-a711-b8b520a00289 None admin] PATH_INFO: `/v3/projects/8ae24634160b4859b608dd29275497f2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.170973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082d139-327f-45a7-a711-b8b520a00289 None admin] RBAC: Authorizing `identity:get_project(project_id=8ae24634160b4859b608dd29275497f2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.172445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082d139-327f-45a7-a711-b8b520a00289 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.174160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 223/439] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/projects/8ae24634160b4859b608dd29275497f2 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:50.174911 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082d139-327f-45a7-a711-b8b520a00289 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.207789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850e08e-5038-4fbc-ae96-e3ea6cc7b917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.208457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850e08e-5038-4fbc-ae96-e3ea6cc7b917 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.208756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850e08e-5038-4fbc-ae96-e3ea6cc7b917 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.209039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850e08e-5038-4fbc-ae96-e3ea6cc7b917 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.209794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850e08e-5038-4fbc-ae96-e3ea6cc7b917 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:50.210104 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850e08e-5038-4fbc-ae96-e3ea6cc7b917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.211477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850e08e-5038-4fbc-ae96-e3ea6cc7b917 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.219524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 217/440] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:50.220698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef2e87-d1d9-4815-8b0e-d2a2687799b4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.221012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef2e87-d1d9-4815-8b0e-d2a2687799b4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.221288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef2e87-d1d9-4815-8b0e-d2a2687799b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.237737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9cef2e87-d1d9-4815-8b0e-d2a2687799b4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:50.239110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9cef2e87-d1d9-4815-8b0e-d2a2687799b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:50.281090 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082d139-327f-45a7-a711-b8b520a00289 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.312923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 218/441] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:50.312923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef2e87-d1d9-4815-8b0e-d2a2687799b4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.315332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62779fd-0d3d-4ab6-98a1-662b8482f788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.315610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62779fd-0d3d-4ab6-98a1-662b8482f788 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.315893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62779fd-0d3d-4ab6-98a1-662b8482f788 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.316169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62779fd-0d3d-4ab6-98a1-662b8482f788 None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.317383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62779fd-0d3d-4ab6-98a1-662b8482f788 None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.318829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62779fd-0d3d-4ab6-98a1-662b8482f788 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.320454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 224/442] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:50.321309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62779fd-0d3d-4ab6-98a1-662b8482f788 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.353744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:50.377958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:50.416333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef2e87-d1d9-4815-8b0e-d2a2687799b4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.426130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62779fd-0d3d-4ab6-98a1-662b8482f788 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.450825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfa06b-cb82-4e63-b65c-56956da91288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.451728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfa06b-cb82-4e63-b65c-56956da91288 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.452008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfa06b-cb82-4e63-b65c-56956da91288 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.452279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfa06b-cb82-4e63-b65c-56956da91288 None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.453489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfa06b-cb82-4e63-b65c-56956da91288 None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.455255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfa06b-cb82-4e63-b65c-56956da91288 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.461696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 219/443] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:50.461696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfa06b-cb82-4e63-b65c-56956da91288 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.466740 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1406a7e-a600-41f4-a3ad-f50fadce46f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.467454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1406a7e-a600-41f4-a3ad-f50fadce46f7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.467716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1406a7e-a600-41f4-a3ad-f50fadce46f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.467986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1406a7e-a600-41f4-a3ad-f50fadce46f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.535019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1406a7e-a600-41f4-a3ad-f50fadce46f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.536393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1406a7e-a600-41f4-a3ad-f50fadce46f7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.555285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfa06b-cb82-4e63-b65c-56956da91288 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.563051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:50.584831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:50.590704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecf23d-5a03-48f4-bd46-a8ef3a267a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.591384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecf23d-5a03-48f4-bd46-a8ef3a267a16 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.591665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecf23d-5a03-48f4-bd46-a8ef3a267a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.591925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecf23d-5a03-48f4-bd46-a8ef3a267a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.592841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ecf23d-5a03-48f4-bd46-a8ef3a267a16 None admin] RBAC: Adding query filter params (group.id=e303cc6ee25c4dc8bd2f29042baab68d, role.id=9ae7e8c79b724c70890cf352b9748307, scope.project.id=46bc7b47b92148ffb91da999460d824f) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:50.593172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ecf23d-5a03-48f4-bd46-a8ef3a267a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.594602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ecf23d-5a03-48f4-bd46-a8ef3a267a16 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.600415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 220/444] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/role_assignments?group.id=e303cc6ee25c4dc8bd2f29042baab68d&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=9ae7e8c79b724c70890cf352b9748307 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:50.601198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecf23d-5a03-48f4-bd46-a8ef3a267a16 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.648673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 225/445] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:50.649579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1406a7e-a600-41f4-a3ad-f50fadce46f7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.720979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecf23d-5a03-48f4-bd46-a8ef3a267a16 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.762434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cfac4-e7c9-4186-9fd8-e7d70fb10eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.763235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cfac4-e7c9-4186-9fd8-e7d70fb10eef None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.763527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cfac4-e7c9-4186-9fd8-e7d70fb10eef None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.763799 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cfac4-e7c9-4186-9fd8-e7d70fb10eef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.765208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1406a7e-a600-41f4-a3ad-f50fadce46f7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.778729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cfac4-e7c9-4186-9fd8-e7d70fb10eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a80cfac4-e7c9-4186-9fd8-e7d70fb10eef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:50.787779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:50.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:50.789505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 221/446] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:50.790141 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cfac4-e7c9-4186-9fd8-e7d70fb10eef None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.798524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:50.809565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375ef58-b802-4e6c-99c1-3db3ed5ef8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.810232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375ef58-b802-4e6c-99c1-3db3ed5ef8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.810546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375ef58-b802-4e6c-99c1-3db3ed5ef8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.810823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375ef58-b802-4e6c-99c1-3db3ed5ef8f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.811820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375ef58-b802-4e6c-99c1-3db3ed5ef8f3 None admin] RBAC: Adding query filter params (group.id=e303cc6ee25c4dc8bd2f29042baab68d, role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=8ae24634160b4859b608dd29275497f2) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:50.812134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375ef58-b802-4e6c-99c1-3db3ed5ef8f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.813655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375ef58-b802-4e6c-99c1-3db3ed5ef8f3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.820570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 226/447] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/role_assignments?group.id=e303cc6ee25c4dc8bd2f29042baab68d&scope.project.id=8ae24634160b4859b608dd29275497f2&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:50.821435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375ef58-b802-4e6c-99c1-3db3ed5ef8f3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.863095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:50.913396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cfac4-e7c9-4186-9fd8-e7d70fb10eef None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.931507 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375ef58-b802-4e6c-99c1-3db3ed5ef8f3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:50.951699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff540c-f622-4060-85b2-82da49178744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.952927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff540c-f622-4060-85b2-82da49178744 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.953222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff540c-f622-4060-85b2-82da49178744 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.953508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff540c-f622-4060-85b2-82da49178744 None admin] PATH_INFO: `/v3/projects/46bc7b47b92148ffb91da999460d824f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.955125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff540c-f622-4060-85b2-82da49178744 None admin] RBAC: Authorizing `identity:get_project(project_id=46bc7b47b92148ffb91da999460d824f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.956611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff540c-f622-4060-85b2-82da49178744 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.958444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 222/448] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/projects/46bc7b47b92148ffb91da999460d824f => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:50.964028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff540c-f622-4060-85b2-82da49178744 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:50.974316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde4dfd-e2b0-4d8d-996a-dfe52f40a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:50.974993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde4dfd-e2b0-4d8d-996a-dfe52f40a8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:50.975278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde4dfd-e2b0-4d8d-996a-dfe52f40a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:50.975555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde4dfd-e2b0-4d8d-996a-dfe52f40a8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:50.976314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde4dfd-e2b0-4d8d-996a-dfe52f40a8b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:50.976619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde4dfd-e2b0-4d8d-996a-dfe52f40a8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:50.978042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde4dfd-e2b0-4d8d-996a-dfe52f40a8b6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:50.986514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 227/449] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:50.992710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde4dfd-e2b0-4d8d-996a-dfe52f40a8b6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:51.057253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:51.079439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff540c-f622-4060-85b2-82da49178744 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:51.107077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:51.112322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde4dfd-e2b0-4d8d-996a-dfe52f40a8b6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:51.119674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce32414-d054-4b21-97b6-bd0bd1f74338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:51.120678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32414-d054-4b21-97b6-bd0bd1f74338 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:51.122018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32414-d054-4b21-97b6-bd0bd1f74338 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:51.122503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32414-d054-4b21-97b6-bd0bd1f74338 None admin] PATH_INFO: `/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:51.123489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce32414-d054-4b21-97b6-bd0bd1f74338 None admin] RBAC: Authorizing `identity:get_group(group_id=e303cc6ee25c4dc8bd2f29042baab68d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:51.124969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce32414-d054-4b21-97b6-bd0bd1f74338 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:51.127988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 223/450] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/groups/e303cc6ee25c4dc8bd2f29042baab68d => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:51.132515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce32414-d054-4b21-97b6-bd0bd1f74338 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:51.149832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5030a7-9657-4cc5-ab3a-9a64b8698f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:51.150489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5030a7-9657-4cc5-ab3a-9a64b8698f20 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:51.150771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5030a7-9657-4cc5-ab3a-9a64b8698f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:51.151048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5030a7-9657-4cc5-ab3a-9a64b8698f20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:51.227951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5030a7-9657-4cc5-ab3a-9a64b8698f20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:51.229848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5030a7-9657-4cc5-ab3a-9a64b8698f20 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:51.260403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce32414-d054-4b21-97b6-bd0bd1f74338 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:51.292395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:51.304842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc79d1a-9138-4ffc-9c82-aa3fe459eb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:51.305510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc79d1a-9138-4ffc-9c82-aa3fe459eb6c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:51.305792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc79d1a-9138-4ffc-9c82-aa3fe459eb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:51.306065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc79d1a-9138-4ffc-9c82-aa3fe459eb6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:51.306909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc79d1a-9138-4ffc-9c82-aa3fe459eb6c None admin] RBAC: Adding query filter params (group.id=e303cc6ee25c4dc8bd2f29042baab68d, role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=46bc7b47b92148ffb91da999460d824f) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:51.307225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc79d1a-9138-4ffc-9c82-aa3fe459eb6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:51.308716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc79d1a-9138-4ffc-9c82-aa3fe459eb6c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:51.314980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 224/451] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:51 2022] GET /identity/v3/role_assignments?group.id=e303cc6ee25c4dc8bd2f29042baab68d&scope.project.id=46bc7b47b92148ffb91da999460d824f&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:22:51.344281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 228/452] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:22:51.350342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc79d1a-9138-4ffc-9c82-aa3fe459eb6c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:51.410299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:51.465151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc79d1a-9138-4ffc-9c82-aa3fe459eb6c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:51.499472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727ae4d-55de-4a8e-a206-cc99472c553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:51.500335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727ae4d-55de-4a8e-a206-cc99472c553c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:51.500610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727ae4d-55de-4a8e-a206-cc99472c553c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:51.500910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727ae4d-55de-4a8e-a206-cc99472c553c None admin] PATH_INFO: `/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:51.502104 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727ae4d-55de-4a8e-a206-cc99472c553c None admin] RBAC: Authorizing `identity:get_project(project_id=0f9dd8794ad7494bbf369d7c8ce42c78)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:51.503543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727ae4d-55de-4a8e-a206-cc99472c553c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:51.505268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 225/453] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:22:51 2022] GET /identity/v3/projects/0f9dd8794ad7494bbf369d7c8ce42c78 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:51.510899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5030a7-9657-4cc5-ab3a-9a64b8698f20 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:51.613149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5030a7-9657-4cc5-ab3a-9a64b8698f20 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:51.646993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f119e6-f48b-4b3c-8af9-216b895b67dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:51.647849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f119e6-f48b-4b3c-8af9-216b895b67dc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:51.648129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f119e6-f48b-4b3c-8af9-216b895b67dc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:51.648403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f119e6-f48b-4b3c-8af9-216b895b67dc None admin] PATH_INFO: `/v3/groups/e8cc7b318e714e968fbb8416575c5a99` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:51.649580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f119e6-f48b-4b3c-8af9-216b895b67dc None admin] RBAC: Authorizing `identity:get_group(group_id=e8cc7b318e714e968fbb8416575c5a99)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:51.651015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f119e6-f48b-4b3c-8af9-216b895b67dc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:51.652642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 229/454] 10.222.0.69 () {56 vars in 1180 bytes} [Fri Jun 24 10:22:51 2022] GET /identity/v3/groups/e8cc7b318e714e968fbb8416575c5a99 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:51.658718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727ae4d-55de-4a8e-a206-cc99472c553c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:51.759430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727ae4d-55de-4a8e-a206-cc99472c553c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:51.783690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:51.797268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942fefa-f51b-4605-ac9c-8daac0263cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:51.798311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942fefa-f51b-4605-ac9c-8daac0263cab None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:51.798666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942fefa-f51b-4605-ac9c-8daac0263cab None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:51.798945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942fefa-f51b-4605-ac9c-8daac0263cab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:51.799794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942fefa-f51b-4605-ac9c-8daac0263cab None admin] RBAC: Adding query filter params (group.id=e8cc7b318e714e968fbb8416575c5a99, role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=0f9dd8794ad7494bbf369d7c8ce42c78) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:51.800107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942fefa-f51b-4605-ac9c-8daac0263cab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:51.801595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942fefa-f51b-4605-ac9c-8daac0263cab None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:51.808129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 226/455] 10.222.0.69 () {56 vars in 1376 bytes} [Fri Jun 24 10:22:51 2022] GET /identity/v3/role_assignments?group.id=e8cc7b318e714e968fbb8416575c5a99&scope.project.id=0f9dd8794ad7494bbf369d7c8ce42c78&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:54.212377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b663e-9bf9-46b0-873b-ef422447455e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.213282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b663e-9bf9-46b0-873b-ef422447455e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.213699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b663e-9bf9-46b0-873b-ef422447455e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.215431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 230/456] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:54.221127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e6c88-3b2d-44a2-8112-63d6f2feba12 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.221915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e6c88-3b2d-44a2-8112-63d6f2feba12 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.222339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e6c88-3b2d-44a2-8112-63d6f2feba12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.243285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-026e6c88-3b2d-44a2-8112-63d6f2feba12 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:54.244689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-026e6c88-3b2d-44a2-8112-63d6f2feba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:54.338229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 227/457] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:54.362505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b663e-9bf9-46b0-873b-ef422447455e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:54.387920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:54.426920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1cec-1fd5-48d7-a9a2-da335c763f54 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.427207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1cec-1fd5-48d7-a9a2-da335c763f54 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.428046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1cec-1fd5-48d7-a9a2-da335c763f54 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.429718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 228/458] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:54.435430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2932-23ee-4e3d-b1fc-8e3a145e6c53 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.436109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2932-23ee-4e3d-b1fc-8e3a145e6c53 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.436621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2932-23ee-4e3d-b1fc-8e3a145e6c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.470915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9d6f2932-23ee-4e3d-b1fc-8e3a145e6c53 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:54.472029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9d6f2932-23ee-4e3d-b1fc-8e3a145e6c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:54.518719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b663e-9bf9-46b0-873b-ef422447455e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:54.573458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad378c-952b-43a8-9629-206ce00023f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:54.574261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad378c-952b-43a8-9629-206ce00023f0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.574541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad378c-952b-43a8-9629-206ce00023f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.574817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad378c-952b-43a8-9629-206ce00023f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.593714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad378c-952b-43a8-9629-206ce00023f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:54.595183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad378c-952b-43a8-9629-206ce00023f0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:54.605471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 229/459] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-97ad378c-952b-43a8-9629-206ce00023f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:22:54.626876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:54.627928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:54.628952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:22:54.628952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:22:54.628952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:54.628952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:54.628952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 231/460] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:54.631347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2932-23ee-4e3d-b1fc-8e3a145e6c53 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:54.639567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad378c-952b-43a8-9629-206ce00023f0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:54.722707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:54.778833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:54.816713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad378c-952b-43a8-9629-206ce00023f0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:54.823529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2932-23ee-4e3d-b1fc-8e3a145e6c53 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:54.857447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973205a6-e03a-4b55-a52b-23eb4a45ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:54.858567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973205a6-e03a-4b55-a52b-23eb4a45ae94 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.859086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973205a6-e03a-4b55-a52b-23eb4a45ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.859620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973205a6-e03a-4b55-a52b-23eb4a45ae94 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.860775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973205a6-e03a-4b55-a52b-23eb4a45ae94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:54.861127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973205a6-e03a-4b55-a52b-23eb4a45ae94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:54.861321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdc0a7-4b93-4957-8ce3-1c0f302ecc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:54.862174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdc0a7-4b93-4957-8ce3-1c0f302ecc99 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.862460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdc0a7-4b93-4957-8ce3-1c0f302ecc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.862733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdc0a7-4b93-4957-8ce3-1c0f302ecc99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.863367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973205a6-e03a-4b55-a52b-23eb4a45ae94 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:54.873797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdc0a7-4b93-4957-8ce3-1c0f302ecc99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:54.875356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdc0a7-4b93-4957-8ce3-1c0f302ecc99 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:54.889020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 232/461] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:54.890567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4c66c-db69-4832-8700-b055201a9a34 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.890860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4c66c-db69-4832-8700-b055201a9a34 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.891141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4c66c-db69-4832-8700-b055201a9a34 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.892559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 233/462] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:54.897036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4c66c-db69-4832-8700-b055201a9a34 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-72fdc0a7-4b93-4957-8ce3-1c0f302ecc99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:22:54.902192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:54.902989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:54.903809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:22:54.903809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:22:54.903809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:54.903809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:54.903809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 230/463] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:54.904980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd5a59-d894-4c5f-be7a-18898d32f6d5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.905396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd5a59-d894-4c5f-be7a-18898d32f6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.905866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd5a59-d894-4c5f-be7a-18898d32f6d5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.907878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 231/464] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:54.909486 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f9147-fcbb-400c-b8a3-2dcb57cd7142 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:54.910241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f9147-fcbb-400c-b8a3-2dcb57cd7142 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:54.911045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f9147-fcbb-400c-b8a3-2dcb57cd7142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:54.927873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-352f9147-fcbb-400c-b8a3-2dcb57cd7142 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:54.929085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-352f9147-fcbb-400c-b8a3-2dcb57cd7142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:54.996400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 232/465] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:54.997531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f9147-fcbb-400c-b8a3-2dcb57cd7142 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:55.006825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4c66c-db69-4832-8700-b055201a9a34 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:55.029171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:55.047004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce85304-9c67-422e-9b1b-11ca59d6bb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:55.047915 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce85304-9c67-422e-9b1b-11ca59d6bb3e None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.048642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce85304-9c67-422e-9b1b-11ca59d6bb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.048953 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce85304-9c67-422e-9b1b-11ca59d6bb3e None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.049660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce85304-9c67-422e-9b1b-11ca59d6bb3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:55.051114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce85304-9c67-422e-9b1b-11ca59d6bb3e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:55.058937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:55.073331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 234/466] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:54 2022] POST /identity/v3/users => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:55.074848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c2365-313f-4929-97d8-828262ac216b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.075136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c2365-313f-4929-97d8-828262ac216b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.075409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c2365-313f-4929-97d8-828262ac216b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.100603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-669c2365-313f-4929-97d8-828262ac216b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:55.101866 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-669c2365-313f-4929-97d8-828262ac216b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:55.119417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f9147-fcbb-400c-b8a3-2dcb57cd7142 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:55.157829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873b2f7-dd71-413c-afe9-80a5dcdbad7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:55.158563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873b2f7-dd71-413c-afe9-80a5dcdbad7a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.158848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873b2f7-dd71-413c-afe9-80a5dcdbad7a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.159174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873b2f7-dd71-413c-afe9-80a5dcdbad7a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.159992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873b2f7-dd71-413c-afe9-80a5dcdbad7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:55.160301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873b2f7-dd71-413c-afe9-80a5dcdbad7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:55.161822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873b2f7-dd71-413c-afe9-80a5dcdbad7a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:55.175560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 233/467] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:55.176497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873b2f7-dd71-413c-afe9-80a5dcdbad7a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:55.202211 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 235/468] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:55.203175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c2365-313f-4929-97d8-828262ac216b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:55.278411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:55.291588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873b2f7-dd71-413c-afe9-80a5dcdbad7a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:55.329164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadf1c5-8e87-4cf2-be42-0e08839b1429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:55.330588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadf1c5-8e87-4cf2-be42-0e08839b1429 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.330869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadf1c5-8e87-4cf2-be42-0e08839b1429 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.331142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadf1c5-8e87-4cf2-be42-0e08839b1429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.337909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c2365-313f-4929-97d8-828262ac216b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:55.345621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faadf1c5-8e87-4cf2-be42-0e08839b1429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:55.347086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faadf1c5-8e87-4cf2-be42-0e08839b1429 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:55.363146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-faadf1c5-8e87-4cf2-be42-0e08839b1429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:22:55.365787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:55.367079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:55.369672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:22:55.369672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:22:55.369672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:55.369672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:55.369672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 234/469] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:55.369672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadf1c5-8e87-4cf2-be42-0e08839b1429 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:55.378093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1e840-0fc6-4f5c-9d50-b431a9b4ff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:55.379140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e840-0fc6-4f5c-9d50-b431a9b4ff79 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.379859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e840-0fc6-4f5c-9d50-b431a9b4ff79 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.380300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e840-0fc6-4f5c-9d50-b431a9b4ff79 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.381196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1e840-0fc6-4f5c-9d50-b431a9b4ff79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:55.382798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1e840-0fc6-4f5c-9d50-b431a9b4ff79 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:55.408351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 236/470] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:55 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:55.413700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1e840-0fc6-4f5c-9d50-b431a9b4ff79 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:55.533993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1e840-0fc6-4f5c-9d50-b431a9b4ff79 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:55.571953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadf1c5-8e87-4cf2-be42-0e08839b1429 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:55.608330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480dc53-7d95-4c50-ab5f-cf1decb2423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:55.609271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480dc53-7d95-4c50-ab5f-cf1decb2423f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.609561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480dc53-7d95-4c50-ab5f-cf1decb2423f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.609843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480dc53-7d95-4c50-ab5f-cf1decb2423f None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.610586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480dc53-7d95-4c50-ab5f-cf1decb2423f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:55.610896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480dc53-7d95-4c50-ab5f-cf1decb2423f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:55.612305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480dc53-7d95-4c50-ab5f-cf1decb2423f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:55.618652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af1858-5561-4d04-b594-e4c9b54e5a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:55.619709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af1858-5561-4d04-b594-e4c9b54e5a86 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.620186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af1858-5561-4d04-b594-e4c9b54e5a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.620626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af1858-5561-4d04-b594-e4c9b54e5a86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.627266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 237/471] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:55.632571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480dc53-7d95-4c50-ab5f-cf1decb2423f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:55.634764 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af1858-5561-4d04-b594-e4c9b54e5a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:55.636716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af1858-5561-4d04-b594-e4c9b54e5a86 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:55.640140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c4af1858-5561-4d04-b594-e4c9b54e5a86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:22:55.661536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:55.662527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:55.663352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:22:55.663352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:22:55.663352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:55.663352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:55.666176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 235/472] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:55.674081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af1858-5561-4d04-b594-e4c9b54e5a86 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:55.683107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:55.771526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480dc53-7d95-4c50-ab5f-cf1decb2423f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:55.807813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c72fa-278c-4f33-91db-36acf78dce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:55.808406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c72fa-278c-4f33-91db-36acf78dce2f None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.808691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c72fa-278c-4f33-91db-36acf78dce2f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.808994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c72fa-278c-4f33-91db-36acf78dce2f None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.809702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c72fa-278c-4f33-91db-36acf78dce2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:55.811176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c72fa-278c-4f33-91db-36acf78dce2f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:55.816778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af1858-5561-4d04-b594-e4c9b54e5a86 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:55.844769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 238/473] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:55 2022] POST /identity/v3/users => generated 249 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:22:55.844769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6e92f-564a-442e-873a-77139ecd422a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.844769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6e92f-564a-442e-873a-77139ecd422a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.844769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6e92f-564a-442e-873a-77139ecd422a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.844769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 239/474] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:55.848306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce0785-2b55-4c9e-8538-a715dfa91ac2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.848633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce0785-2b55-4c9e-8538-a715dfa91ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.850273 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce0785-2b55-4c9e-8538-a715dfa91ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.882831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e67be4-2e45-429f-999b-9ffb84e75ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:55.883513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e67be4-2e45-429f-999b-9ffb84e75ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:55.883801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e67be4-2e45-429f-999b-9ffb84e75ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:55.883929 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f4ce0785-2b55-4c9e-8538-a715dfa91ac2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:55.884080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e67be4-2e45-429f-999b-9ffb84e75ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:55.884847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e67be4-2e45-429f-999b-9ffb84e75ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:55.885035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f4ce0785-2b55-4c9e-8538-a715dfa91ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:55.885187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e67be4-2e45-429f-999b-9ffb84e75ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:55.886606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e67be4-2e45-429f-999b-9ffb84e75ea6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:55.901005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 236/475] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:55.908912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e67be4-2e45-429f-999b-9ffb84e75ea6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:56.027003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:56.033026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 240/476] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:56.063374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce0785-2b55-4c9e-8538-a715dfa91ac2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:56.091591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:56.105910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e67be4-2e45-429f-999b-9ffb84e75ea6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:56.143740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a1928-d25b-4d58-a889-280263d6a8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:56.143946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a1928-d25b-4d58-a889-280263d6a8d7 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:56.144165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a1928-d25b-4d58-a889-280263d6a8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:56.144417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a1928-d25b-4d58-a889-280263d6a8d7 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:56.145177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a1928-d25b-4d58-a889-280263d6a8d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:56.146630 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a1928-d25b-4d58-a889-280263d6a8d7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:56.176200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 237/477] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:22:55 2022] POST /identity/v3/users => generated 248 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:22:56.184455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce0785-2b55-4c9e-8538-a715dfa91ac2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:56.227592 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:56.228650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:56.229169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:56.229627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:56.230595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:56.232269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:56.239676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.sql.core [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 10:22:56.239676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 10:22:56.239676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [parameters: {'id': '863ed77d1a424eb594e425e701af8b79', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 10:22:56.239676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80145) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application result.read() Jun 24 10:22:56.257822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:22:56.258734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application yield resource Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise self.value Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application session.commit() Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 10:22:56.259614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.session.flush() Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application compat.raise_( Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 10:22:56.260427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:22:56.261310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application result.read() Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:22:56.262098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [parameters: {'id': '863ed77d1a424eb594e425e701af8b79', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:56.262857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 10:22:56.263628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:56.267811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 241/478] 10.222.0.69 () {60 vars in 1131 bytes} [Fri Jun 24 10:22:56 2022] POST /identity/v3/roles => generated 144 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:56.273461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a1928-d25b-4d58-a889-280263d6a8d7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:56.397179 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a1928-d25b-4d58-a889-280263d6a8d7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:56.412773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:56.460899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca018f-0376-4896-8aa3-fc7b79ee54d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:56.461759 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca018f-0376-4896-8aa3-fc7b79ee54d7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:56.462041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca018f-0376-4896-8aa3-fc7b79ee54d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:56.462313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca018f-0376-4896-8aa3-fc7b79ee54d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:56.481175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca018f-0376-4896-8aa3-fc7b79ee54d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a5ca018f-0376-4896-8aa3-fc7b79ee54d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:56.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:56.492771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:56.493555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 238/479] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:22:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:56.498787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:56.589086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:56.604543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cad0e-d669-4cf3-93c4-4701efa0465d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:56.639261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe6211-421f-4059-ac17-b88cfe849f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:56.640269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe6211-421f-4059-ac17-b88cfe849f59 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:56.640761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe6211-421f-4059-ac17-b88cfe849f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:56.641227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe6211-421f-4059-ac17-b88cfe849f59 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:56.642185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe6211-421f-4059-ac17-b88cfe849f59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:56.642682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe6211-421f-4059-ac17-b88cfe849f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:56.644262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe6211-421f-4059-ac17-b88cfe849f59 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:56.653212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 242/480] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:22:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:57.455617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822e61-b361-4b62-b7e5-c7691f1631a4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:57.456652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822e61-b361-4b62-b7e5-c7691f1631a4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:57.457163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822e61-b361-4b62-b7e5-c7691f1631a4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:57.459548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 239/481] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:57.464652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5a554-9840-4d78-ab42-20dc115e1e5f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:57.465169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5a554-9840-4d78-ab42-20dc115e1e5f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:57.465651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5a554-9840-4d78-ab42-20dc115e1e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:57.487043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-54c5a554-9840-4d78-ab42-20dc115e1e5f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:57.488552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-54c5a554-9840-4d78-ab42-20dc115e1e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:57.575487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 243/482] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:57.619126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf822e61-b361-4b62-b7e5-c7691f1631a4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:57.726405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf822e61-b361-4b62-b7e5-c7691f1631a4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:57.762636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b6d71-01b0-40f6-85de-7b8753a8b954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:57.763543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b6d71-01b0-40f6-85de-7b8753a8b954 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:57.763826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b6d71-01b0-40f6-85de-7b8753a8b954 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:57.764128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b6d71-01b0-40f6-85de-7b8753a8b954 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:57.774542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b6d71-01b0-40f6-85de-7b8753a8b954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a14b6d71-01b0-40f6-85de-7b8753a8b954 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:57.782585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:57.783445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:22:57.784214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:57.790332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 240/483] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:22:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:57.795835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5a554-9840-4d78-ab42-20dc115e1e5f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:57.910175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:57.932010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5a554-9840-4d78-ab42-20dc115e1e5f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:57.967497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b65fd0-5aef-4bae-abc1-84bf1e63ab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:57.968174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b65fd0-5aef-4bae-abc1-84bf1e63ab0a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:57.968459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b65fd0-5aef-4bae-abc1-84bf1e63ab0a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:57.968752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b65fd0-5aef-4bae-abc1-84bf1e63ab0a None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:57.969519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b65fd0-5aef-4bae-abc1-84bf1e63ab0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:57.969829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b65fd0-5aef-4bae-abc1-84bf1e63ab0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:57.971331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b65fd0-5aef-4bae-abc1-84bf1e63ab0a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:57.980187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 244/484] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:22:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:57.985560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b6d71-01b0-40f6-85de-7b8753a8b954 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:57.993804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9586a5-4e35-429a-b659-9f104857c71d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:57.994089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9586a5-4e35-429a-b659-9f104857c71d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:57.994372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9586a5-4e35-429a-b659-9f104857c71d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:57.996692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 245/485] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:22:58.008757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4196c08-5a51-42bf-81e9-3cd635045701 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.008757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4196c08-5a51-42bf-81e9-3cd635045701 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.008757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4196c08-5a51-42bf-81e9-3cd635045701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.022792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a4196c08-5a51-42bf-81e9-3cd635045701 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:58.024468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a4196c08-5a51-42bf-81e9-3cd635045701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:58.058546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:58.096537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 246/486] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:22:58.123144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4196c08-5a51-42bf-81e9-3cd635045701 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:58.220439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b6d71-01b0-40f6-85de-7b8753a8b954 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:58.238839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:58.287421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d5d02-b105-4c9f-b683-c07383380939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:58.289586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d5d02-b105-4c9f-b683-c07383380939 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.290693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d5d02-b105-4c9f-b683-c07383380939 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.291723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d5d02-b105-4c9f-b683-c07383380939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.298002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4196c08-5a51-42bf-81e9-3cd635045701 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:58.316173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d5d02-b105-4c9f-b683-c07383380939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:58.317950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d5d02-b105-4c9f-b683-c07383380939 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-0a9d5d02-b105-4c9f-b683-c07383380939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:22:58.348266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:58.349159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:58.350018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:22:58.350018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:22:58.350018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:58.350018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:58.357811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 241/487] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:58.362316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d5d02-b105-4c9f-b683-c07383380939 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:58.369232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4fbd4-f418-47c1-a14e-899857999cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:58.370066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4fbd4-f418-47c1-a14e-899857999cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.370351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4fbd4-f418-47c1-a14e-899857999cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.370626 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4fbd4-f418-47c1-a14e-899857999cf2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.389808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4fbd4-f418-47c1-a14e-899857999cf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-33f4fbd4-f418-47c1-a14e-899857999cf2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:58.409872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:58.410949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:22:58.411906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:58.413093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 247/488] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:22:58.422705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4fbd4-f418-47c1-a14e-899857999cf2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:58.452198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:58.474051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d5d02-b105-4c9f-b683-c07383380939 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:58.511129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b3ef5-2cfa-4113-8e65-731d86cefa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:58.512115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b3ef5-2cfa-4113-8e65-731d86cefa1a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.513805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b3ef5-2cfa-4113-8e65-731d86cefa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.514305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b3ef5-2cfa-4113-8e65-731d86cefa1a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.515763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b3ef5-2cfa-4113-8e65-731d86cefa1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:58.516434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b3ef5-2cfa-4113-8e65-731d86cefa1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:58.518680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b3ef5-2cfa-4113-8e65-731d86cefa1a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:58.534333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 242/489] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:58.536626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf768f-cb9e-4f8c-a3c0-26baf1ac050f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.537406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf768f-cb9e-4f8c-a3c0-26baf1ac050f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.537879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf768f-cb9e-4f8c-a3c0-26baf1ac050f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.543807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 243/490] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:58.543807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf768f-cb9e-4f8c-a3c0-26baf1ac050f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:58.603784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4fbd4-f418-47c1-a14e-899857999cf2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:58.635825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:58.643284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664616c0-f6de-45cd-b89a-75307da82d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:58.644262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664616c0-f6de-45cd-b89a-75307da82d27 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.644763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664616c0-f6de-45cd-b89a-75307da82d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.645237 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664616c0-f6de-45cd-b89a-75307da82d27 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.646174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf768f-cb9e-4f8c-a3c0-26baf1ac050f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:58.646451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664616c0-f6de-45cd-b89a-75307da82d27 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:58.646997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664616c0-f6de-45cd-b89a-75307da82d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:58.648907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664616c0-f6de-45cd-b89a-75307da82d27 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:58.658853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 248/491] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:22:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:58.660492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d82b88-c30b-44ef-afd8-09d1c3ba83f2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.661058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d82b88-c30b-44ef-afd8-09d1c3ba83f2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.661650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d82b88-c30b-44ef-afd8-09d1c3ba83f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.679340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e9d82b88-c30b-44ef-afd8-09d1c3ba83f2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:58.680573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e9d82b88-c30b-44ef-afd8-09d1c3ba83f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:58.689379 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c785b-a408-42c3-b555-9c9e8b658293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:58.690485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c785b-a408-42c3-b555-9c9e8b658293 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.691468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c785b-a408-42c3-b555-9c9e8b658293 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.691915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c785b-a408-42c3-b555-9c9e8b658293 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.702187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c785b-a408-42c3-b555-9c9e8b658293 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:58.704685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c785b-a408-42c3-b555-9c9e8b658293 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-398c785b-a408-42c3-b555-9c9e8b658293 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:22:58.726384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:58.727319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:22:58.728187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:58.730686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 244/492] 10.222.0.69 () {56 vars in 1093 bytes} [Fri Jun 24 10:22:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:58.731752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c785b-a408-42c3-b555-9c9e8b658293 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:58.736158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:58.764982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 249/493] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:58.765966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d82b88-c30b-44ef-afd8-09d1c3ba83f2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:58.864753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c785b-a408-42c3-b555-9c9e8b658293 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:58.882159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d82b88-c30b-44ef-afd8-09d1c3ba83f2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:58.905829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:58.910003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc9dd3-514e-44e1-a875-084a9f88f5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:58.911174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9dd3-514e-44e1-a875-084a9f88f5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.911939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9dd3-514e-44e1-a875-084a9f88f5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.912362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9dd3-514e-44e1-a875-084a9f88f5a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.919574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31578412-35d6-4a85-97c2-b83b0fba1d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:58.920440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31578412-35d6-4a85-97c2-b83b0fba1d5c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.920854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31578412-35d6-4a85-97c2-b83b0fba1d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.921223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31578412-35d6-4a85-97c2-b83b0fba1d5c None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.922177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31578412-35d6-4a85-97c2-b83b0fba1d5c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:58.922579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31578412-35d6-4a85-97c2-b83b0fba1d5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:58.924125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31578412-35d6-4a85-97c2-b83b0fba1d5c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:58.926102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc9dd3-514e-44e1-a875-084a9f88f5a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:58.927870 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc9dd3-514e-44e1-a875-084a9f88f5a5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-dffc9dd3-514e-44e1-a875-084a9f88f5a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:22:58.947937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:58.948995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:58.950028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:22:58.950028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:22:58.950028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:58.950028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:58.952051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 245/494] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:58.953618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc9dd3-514e-44e1-a875-084a9f88f5a5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:58.966776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 250/495] 10.222.0.69 () {56 vars in 1133 bytes} [Fri Jun 24 10:22:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:58.968055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31578412-35d6-4a85-97c2-b83b0fba1d5c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.023214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:59.074663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31578412-35d6-4a85-97c2-b83b0fba1d5c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.077676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc9dd3-514e-44e1-a875-084a9f88f5a5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.108022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e10972-48bc-4bcd-9035-f3c73cd6d952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.108689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e10972-48bc-4bcd-9035-f3c73cd6d952 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.109005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e10972-48bc-4bcd-9035-f3c73cd6d952 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.109283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e10972-48bc-4bcd-9035-f3c73cd6d952 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.110004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e10972-48bc-4bcd-9035-f3c73cd6d952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:59.110309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e10972-48bc-4bcd-9035-f3c73cd6d952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:59.111701 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e10972-48bc-4bcd-9035-f3c73cd6d952 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:59.115283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687442be-8c86-417a-b346-96bbd90cd0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.116327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687442be-8c86-417a-b346-96bbd90cd0ed None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.116806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687442be-8c86-417a-b346-96bbd90cd0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.117254 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687442be-8c86-417a-b346-96bbd90cd0ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.123481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 251/496] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:59.124317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e10972-48bc-4bcd-9035-f3c73cd6d952 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.133756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687442be-8c86-417a-b346-96bbd90cd0ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-687442be-8c86-417a-b346-96bbd90cd0ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:59.141939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:59.142804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:22:59.143566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:59.145612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 246/497] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:59.147080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc702a4-fd6c-4b25-8528-c5a37de81137 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.147500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc702a4-fd6c-4b25-8528-c5a37de81137 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.147928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc702a4-fd6c-4b25-8528-c5a37de81137 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.149344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 247/498] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:59.150317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc702a4-fd6c-4b25-8528-c5a37de81137 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.178915 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:59.239605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e10972-48bc-4bcd-9035-f3c73cd6d952 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.258447 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc702a4-fd6c-4b25-8528-c5a37de81137 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.272327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:59.273122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4929ad8-1462-400d-aacd-216ba38e21df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.273824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4929ad8-1462-400d-aacd-216ba38e21df None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.274055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4929ad8-1462-400d-aacd-216ba38e21df None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.274329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4929ad8-1462-400d-aacd-216ba38e21df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.295570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec037562-3eb0-476e-90bf-8347c9fa5a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.296371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec037562-3eb0-476e-90bf-8347c9fa5a46 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.296653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec037562-3eb0-476e-90bf-8347c9fa5a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.296957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec037562-3eb0-476e-90bf-8347c9fa5a46 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.304678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec037562-3eb0-476e-90bf-8347c9fa5a46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:59.306161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec037562-3eb0-476e-90bf-8347c9fa5a46 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ec037562-3eb0-476e-90bf-8347c9fa5a46 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:22:59.320427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:59.321356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 248/499] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:22:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:59.322824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec037562-3eb0-476e-90bf-8347c9fa5a46 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.348287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4929ad8-1462-400d-aacd-216ba38e21df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:59.349769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4929ad8-1462-400d-aacd-216ba38e21df None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:59.390457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:59.426381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec037562-3eb0-476e-90bf-8347c9fa5a46 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.451011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 252/500] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:22:59.454407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c46ff-77e6-411f-9d92-d712685b21dc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.454407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c46ff-77e6-411f-9d92-d712685b21dc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.454407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c46ff-77e6-411f-9d92-d712685b21dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.465994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb59e04-c2d0-478c-9b5d-1cd6f13f8d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.466667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb59e04-c2d0-478c-9b5d-1cd6f13f8d72 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.466947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb59e04-c2d0-478c-9b5d-1cd6f13f8d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.467385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb59e04-c2d0-478c-9b5d-1cd6f13f8d72 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.468153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb59e04-c2d0-478c-9b5d-1cd6f13f8d72 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:59.468458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb59e04-c2d0-478c-9b5d-1cd6f13f8d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:59.469502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7b1c46ff-77e6-411f-9d92-d712685b21dc None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:59.469874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb59e04-c2d0-478c-9b5d-1cd6f13f8d72 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:59.470685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7b1c46ff-77e6-411f-9d92-d712685b21dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:59.477904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 249/501] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:22:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:59.479126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223765d9-f10b-47f5-9acd-b29cbfd778f2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.479416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223765d9-f10b-47f5-9acd-b29cbfd778f2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.479692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223765d9-f10b-47f5-9acd-b29cbfd778f2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.483041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 250/502] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:59.483041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223765d9-f10b-47f5-9acd-b29cbfd778f2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.518178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:59.543181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 253/503] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:59.543956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c46ff-77e6-411f-9d92-d712685b21dc None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.591612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223765d9-f10b-47f5-9acd-b29cbfd778f2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.613879 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:59.625041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef29b3-cd8f-42ed-9435-61dcf4619c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.625711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef29b3-cd8f-42ed-9435-61dcf4619c50 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.625991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef29b3-cd8f-42ed-9435-61dcf4619c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.626283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef29b3-cd8f-42ed-9435-61dcf4619c50 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.627123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef29b3-cd8f-42ed-9435-61dcf4619c50 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:59.627412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef29b3-cd8f-42ed-9435-61dcf4619c50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:59.628878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef29b3-cd8f-42ed-9435-61dcf4619c50 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:59.645351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c46ff-77e6-411f-9d92-d712685b21dc None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.664911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 251/504] 10.222.0.69 () {56 vars in 1137 bytes} [Fri Jun 24 10:22:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:59.665752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef29b3-cd8f-42ed-9435-61dcf4619c50 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.679583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7890654-a7d4-4d34-b08b-e7569e5d9703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.680384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890654-a7d4-4d34-b08b-e7569e5d9703 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.680665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890654-a7d4-4d34-b08b-e7569e5d9703 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.680978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890654-a7d4-4d34-b08b-e7569e5d9703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.690839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7890654-a7d4-4d34-b08b-e7569e5d9703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:59.692300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7890654-a7d4-4d34-b08b-e7569e5d9703 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-e7890654-a7d4-4d34-b08b-e7569e5d9703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:22:59.709753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:59.710573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:59.711367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:22:59.711367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:22:59.711367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:59.711367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:59.711367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 254/505] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:59.712170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e8b54-a6e7-4eb6-af20-382cd531a165 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.712450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e8b54-a6e7-4eb6-af20-382cd531a165 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.712722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e8b54-a6e7-4eb6-af20-382cd531a165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.729443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3d6e8b54-a6e7-4eb6-af20-382cd531a165 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:59.730642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3d6e8b54-a6e7-4eb6-af20-382cd531a165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:59.764437 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:59.771292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef29b3-cd8f-42ed-9435-61dcf4619c50 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.796416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 255/506] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:22:59.797214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e8b54-a6e7-4eb6-af20-382cd531a165 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.805776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac07511-09c8-45a3-a026-771e21386479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.806926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac07511-09c8-45a3-a026-771e21386479 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.807218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac07511-09c8-45a3-a026-771e21386479 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.807499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac07511-09c8-45a3-a026-771e21386479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.817937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac07511-09c8-45a3-a026-771e21386479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:59.819403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac07511-09c8-45a3-a026-771e21386479 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-9ac07511-09c8-45a3-a026-771e21386479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:22:59.837659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:59.838517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:59.839343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:22:59.839343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:22:59.839343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:22:59.839343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:22:59.839343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 252/507] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:59.839660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac07511-09c8-45a3-a026-771e21386479 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.851853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:59.900153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e8b54-a6e7-4eb6-af20-382cd531a165 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.933830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916df31-8485-48ef-82d8-7522c9bc5962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.934631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916df31-8485-48ef-82d8-7522c9bc5962 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.934915 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916df31-8485-48ef-82d8-7522c9bc5962 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.935186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916df31-8485-48ef-82d8-7522c9bc5962 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.939946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac07511-09c8-45a3-a026-771e21386479 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:59.946351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916df31-8485-48ef-82d8-7522c9bc5962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-1916df31-8485-48ef-82d8-7522c9bc5962 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:22:59.954133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:22:59.955555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 256/508] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:22:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:22:59.956508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916df31-8485-48ef-82d8-7522c9bc5962 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:59.978413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f66e6-ca13-4121-a088-d715efbf5c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:59.979059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f66e6-ca13-4121-a088-d715efbf5c27 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.979341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f66e6-ca13-4121-a088-d715efbf5c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.979617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f66e6-ca13-4121-a088-d715efbf5c27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.994497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:00.045509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f66e6-ca13-4121-a088-d715efbf5c27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.047047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f66e6-ca13-4121-a088-d715efbf5c27 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.054262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916df31-8485-48ef-82d8-7522c9bc5962 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.072376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:00.098216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564f648-c4b6-42e6-b9cd-8e3c8991fe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.098883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564f648-c4b6-42e6-b9cd-8e3c8991fe79 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.099163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564f648-c4b6-42e6-b9cd-8e3c8991fe79 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.099790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564f648-c4b6-42e6-b9cd-8e3c8991fe79 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.100154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564f648-c4b6-42e6-b9cd-8e3c8991fe79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:00.100554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564f648-c4b6-42e6-b9cd-8e3c8991fe79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.101911 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564f648-c4b6-42e6-b9cd-8e3c8991fe79 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.113894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 257/509] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:00.114491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564f648-c4b6-42e6-b9cd-8e3c8991fe79 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.154818 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 253/510] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:22:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:00.155683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f66e6-ca13-4121-a088-d715efbf5c27 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.212051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:00.220076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564f648-c4b6-42e6-b9cd-8e3c8991fe79 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.254421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdccf3-7c6e-4494-a7c8-485c97779e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.255222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdccf3-7c6e-4494-a7c8-485c97779e99 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.255506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdccf3-7c6e-4494-a7c8-485c97779e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.255781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdccf3-7c6e-4494-a7c8-485c97779e99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.265838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdccf3-7c6e-4494-a7c8-485c97779e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.267465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdccf3-7c6e-4494-a7c8-485c97779e99 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.274849 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f66e6-ca13-4121-a088-d715efbf5c27 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-31bdccf3-7c6e-4494-a7c8-485c97779e99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:00.285943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:00.286985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:00.292841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:00.292841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:00.292841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:00.292841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:00.292841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 258/511] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:00.292841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdccf3-7c6e-4494-a7c8-485c97779e99 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.314520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:00.319512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2414d5-4106-45d4-93aa-f3b91dd0e1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.320197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2414d5-4106-45d4-93aa-f3b91dd0e1af None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.320487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2414d5-4106-45d4-93aa-f3b91dd0e1af None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.320792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2414d5-4106-45d4-93aa-f3b91dd0e1af None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.321650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2414d5-4106-45d4-93aa-f3b91dd0e1af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:00.321837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2414d5-4106-45d4-93aa-f3b91dd0e1af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.323231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2414d5-4106-45d4-93aa-f3b91dd0e1af None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.336487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 254/512] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:00.337233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2414d5-4106-45d4-93aa-f3b91dd0e1af None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.398796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdccf3-7c6e-4494-a7c8-485c97779e99 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.432850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c143b-300c-485b-baa2-20feaff78017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.433522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c143b-300c-485b-baa2-20feaff78017 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.433799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c143b-300c-485b-baa2-20feaff78017 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.434073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c143b-300c-485b-baa2-20feaff78017 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.434815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c143b-300c-485b-baa2-20feaff78017 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:00.435122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c143b-300c-485b-baa2-20feaff78017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.436570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c143b-300c-485b-baa2-20feaff78017 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.438735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2414d5-4106-45d4-93aa-f3b91dd0e1af None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.444665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 259/513] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:00.445661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c143b-300c-485b-baa2-20feaff78017 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.445868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:00.476640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12b058-34bb-4150-9eaf-7a5809a13ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.477692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12b058-34bb-4150-9eaf-7a5809a13ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.478117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12b058-34bb-4150-9eaf-7a5809a13ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.478647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12b058-34bb-4150-9eaf-7a5809a13ed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.485388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12b058-34bb-4150-9eaf-7a5809a13ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.486814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12b058-34bb-4150-9eaf-7a5809a13ed9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ec12b058-34bb-4150-9eaf-7a5809a13ed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:00.501480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:00.502499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 255/514] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:00.503552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12b058-34bb-4150-9eaf-7a5809a13ed9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.533205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:00.548145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c143b-300c-485b-baa2-20feaff78017 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.582004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143019d8-8fe7-4fd3-a47b-49a9b3be045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.582725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143019d8-8fe7-4fd3-a47b-49a9b3be045d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.583007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143019d8-8fe7-4fd3-a47b-49a9b3be045d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.583284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143019d8-8fe7-4fd3-a47b-49a9b3be045d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.593841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143019d8-8fe7-4fd3-a47b-49a9b3be045d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.595300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143019d8-8fe7-4fd3-a47b-49a9b3be045d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.601705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12b058-34bb-4150-9eaf-7a5809a13ed9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-143019d8-8fe7-4fd3-a47b-49a9b3be045d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:00.614097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:00.615233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:00.616039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:00.616039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:00.616039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:00.616039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:00.616039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 260/515] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:00.616039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143019d8-8fe7-4fd3-a47b-49a9b3be045d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.638991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f56f4-e668-4f31-83a3-1d93ec2bb394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.639654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f56f4-e668-4f31-83a3-1d93ec2bb394 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.639932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f56f4-e668-4f31-83a3-1d93ec2bb394 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.640205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f56f4-e668-4f31-83a3-1d93ec2bb394 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.640988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f56f4-e668-4f31-83a3-1d93ec2bb394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:00.641283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f56f4-e668-4f31-83a3-1d93ec2bb394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.642647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f56f4-e668-4f31-83a3-1d93ec2bb394 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.653689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 256/516] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:00.654468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f56f4-e668-4f31-83a3-1d93ec2bb394 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.681059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:00.715842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143019d8-8fe7-4fd3-a47b-49a9b3be045d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.750372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a670a-ce29-4a0d-a6c0-5993e4940edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.751173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a670a-ce29-4a0d-a6c0-5993e4940edc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.751455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a670a-ce29-4a0d-a6c0-5993e4940edc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.751730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a670a-ce29-4a0d-a6c0-5993e4940edc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.753294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f56f4-e668-4f31-83a3-1d93ec2bb394 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.759084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a670a-ce29-4a0d-a6c0-5993e4940edc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.760545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a670a-ce29-4a0d-a6c0-5993e4940edc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.762291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4f3a670a-ce29-4a0d-a6c0-5993e4940edc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:00.777315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:00.778321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:00.779183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 261/517] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:00.780289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a670a-ce29-4a0d-a6c0-5993e4940edc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.789418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6e261-4071-47f3-b342-ace07d6a5816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.790244 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6e261-4071-47f3-b342-ace07d6a5816 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.790523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6e261-4071-47f3-b342-ace07d6a5816 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.790802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6e261-4071-47f3-b342-ace07d6a5816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.801738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6e261-4071-47f3-b342-ace07d6a5816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.803200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6e261-4071-47f3-b342-ace07d6a5816 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5da6e261-4071-47f3-b342-ace07d6a5816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:00.820082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:00.820988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:00.821834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:00.821834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:00.821834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:00.821834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:00.821834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 257/518] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:00.822359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6e261-4071-47f3-b342-ace07d6a5816 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.881153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a670a-ce29-4a0d-a6c0-5993e4940edc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.899813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:00.914856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be866464-f539-4065-8deb-f8b3538ded0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.915516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be866464-f539-4065-8deb-f8b3538ded0e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.915798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be866464-f539-4065-8deb-f8b3538ded0e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.916073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be866464-f539-4065-8deb-f8b3538ded0e None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.916906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be866464-f539-4065-8deb-f8b3538ded0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:00.917216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be866464-f539-4065-8deb-f8b3538ded0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.918656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be866464-f539-4065-8deb-f8b3538ded0e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.922062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6e261-4071-47f3-b342-ace07d6a5816 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:00.935870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 262/519] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:00.936752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be866464-f539-4065-8deb-f8b3538ded0e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.957611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83341231-2121-483b-94bd-fa3ed5691a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:00.958276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83341231-2121-483b-94bd-fa3ed5691a9b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:00.958558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83341231-2121-483b-94bd-fa3ed5691a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:00.958832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83341231-2121-483b-94bd-fa3ed5691a9b None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:00.959563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83341231-2121-483b-94bd-fa3ed5691a9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:00.959872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83341231-2121-483b-94bd-fa3ed5691a9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:00.961295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83341231-2121-483b-94bd-fa3ed5691a9b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:00.974043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 258/520] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:00.974838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83341231-2121-483b-94bd-fa3ed5691a9b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:00.994372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:01.035660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be866464-f539-4065-8deb-f8b3538ded0e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:01.070830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a895df1-fc46-48e5-b6ca-d49cb6c4ce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:01.071505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a895df1-fc46-48e5-b6ca-d49cb6c4ce3c None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:01.071795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a895df1-fc46-48e5-b6ca-d49cb6c4ce3c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:01.072081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a895df1-fc46-48e5-b6ca-d49cb6c4ce3c None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:01.072198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83341231-2121-483b-94bd-fa3ed5691a9b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:01.072928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a895df1-fc46-48e5-b6ca-d49cb6c4ce3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:01.074395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a895df1-fc46-48e5-b6ca-d49cb6c4ce3c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:01.105329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 263/521] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:23:00 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:01.105329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a895df1-fc46-48e5-b6ca-d49cb6c4ce3c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:01.120130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401430e-fc1f-421a-a264-c6b24c61db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:01.121572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401430e-fc1f-421a-a264-c6b24c61db7d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:01.121927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401430e-fc1f-421a-a264-c6b24c61db7d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:01.122278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401430e-fc1f-421a-a264-c6b24c61db7d None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:01.123546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401430e-fc1f-421a-a264-c6b24c61db7d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:01.124645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401430e-fc1f-421a-a264-c6b24c61db7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:01.126921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401430e-fc1f-421a-a264-c6b24c61db7d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:01.138790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:01.158675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 259/522] 10.222.0.69 () {56 vars in 1139 bytes} [Fri Jun 24 10:23:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:01.158675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401430e-fc1f-421a-a264-c6b24c61db7d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:01.212329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a895df1-fc46-48e5-b6ca-d49cb6c4ce3c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:01.228550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:01.254390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4ba22-5778-4e42-846f-930725a7e59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:01.255065 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4ba22-5778-4e42-846f-930725a7e59a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:01.255346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4ba22-5778-4e42-846f-930725a7e59a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:01.255621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4ba22-5778-4e42-846f-930725a7e59a None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:01.256707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4ba22-5778-4e42-846f-930725a7e59a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:01.257101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4ba22-5778-4e42-846f-930725a7e59a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:01.258814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4ba22-5778-4e42-846f-930725a7e59a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:01.271794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 264/523] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:01.272708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4ba22-5778-4e42-846f-930725a7e59a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:01.290843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401430e-fc1f-421a-a264-c6b24c61db7d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:01.331369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9446e9-a7c9-487c-a90a-08da5e14c6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:01.332038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9446e9-a7c9-487c-a90a-08da5e14c6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:01.332314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9446e9-a7c9-487c-a90a-08da5e14c6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:01.332595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9446e9-a7c9-487c-a90a-08da5e14c6b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:01.333487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9446e9-a7c9-487c-a90a-08da5e14c6b7 None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=bec9fee282a64933aa0f6fbdd91a93c4) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:01.333805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9446e9-a7c9-487c-a90a-08da5e14c6b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:01.335246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9446e9-a7c9-487c-a90a-08da5e14c6b7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:01.349241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 260/524] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:01 2022] GET /identity/v3/role_assignments?user.id=bec9fee282a64933aa0f6fbdd91a93c4&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:01.349241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9446e9-a7c9-487c-a90a-08da5e14c6b7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:01.400823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4ba22-5778-4e42-846f-930725a7e59a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:01.415904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:01.445492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5284461-527f-43fd-b7dd-87803b728d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:01.447303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5284461-527f-43fd-b7dd-87803b728d74 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:01.447768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5284461-527f-43fd-b7dd-87803b728d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:01.448191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5284461-527f-43fd-b7dd-87803b728d74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:01.459986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5284461-527f-43fd-b7dd-87803b728d74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:01.462782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5284461-527f-43fd-b7dd-87803b728d74 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:01.464425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9446e9-a7c9-487c-a90a-08da5e14c6b7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a5284461-527f-43fd-b7dd-87803b728d74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:01.484557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:01.485651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:01.486611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:01.486611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:01.486611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:01.486611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:01.486611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:01.486611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:01.486611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:01.486611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:01.489456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 265/525] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:01.490565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5284461-527f-43fd-b7dd-87803b728d74 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:01.518818 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9bb85-0c65-4779-9935-1efc314fe857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:01.520189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9bb85-0c65-4779-9935-1efc314fe857 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:01.520493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9bb85-0c65-4779-9935-1efc314fe857 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:01.520776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9bb85-0c65-4779-9935-1efc314fe857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:01.548050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:01.597317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9bb85-0c65-4779-9935-1efc314fe857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:01.598790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9bb85-0c65-4779-9935-1efc314fe857 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:01.637509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5284461-527f-43fd-b7dd-87803b728d74 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:01.660418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:01.685380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aa674-cb9d-40a6-8415-399c2f053c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:01.687255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aa674-cb9d-40a6-8415-399c2f053c0d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:01.687837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aa674-cb9d-40a6-8415-399c2f053c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:01.688349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aa674-cb9d-40a6-8415-399c2f053c0d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:01.698753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1aa674-cb9d-40a6-8415-399c2f053c0d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:01.700216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1aa674-cb9d-40a6-8415-399c2f053c0d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:01.717762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 261/526] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:01.718726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9bb85-0c65-4779-9935-1efc314fe857 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-fe1aa674-cb9d-40a6-8415-399c2f053c0d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:01.719034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:01.719940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:23:01.720831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:01.723994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 266/527] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:01.725047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aa674-cb9d-40a6-8415-399c2f053c0d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:01.827235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:01.834981 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aa674-cb9d-40a6-8415-399c2f053c0d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:01.845483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9bb85-0c65-4779-9935-1efc314fe857 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:01.879411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158251c-6064-4ed8-8402-07c7e5746bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:01.880970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158251c-6064-4ed8-8402-07c7e5746bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:01.881564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158251c-6064-4ed8-8402-07c7e5746bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:01.882126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158251c-6064-4ed8-8402-07c7e5746bf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:01.885961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f982-f0a3-4a43-92f3-1925747fdaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:01.886699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f982-f0a3-4a43-92f3-1925747fdaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:01.887044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f982-f0a3-4a43-92f3-1925747fdaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:01.887397 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f982-f0a3-4a43-92f3-1925747fdaf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:01.888259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97f982-f0a3-4a43-92f3-1925747fdaf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:01.888630 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97f982-f0a3-4a43-92f3-1925747fdaf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:01.890257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97f982-f0a3-4a43-92f3-1925747fdaf7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:01.896020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158251c-6064-4ed8-8402-07c7e5746bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:01.897749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158251c-6064-4ed8-8402-07c7e5746bf5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:01.910114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 262/528] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:01.911105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f982-f0a3-4a43-92f3-1925747fdaf7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a158251c-6064-4ed8-8402-07c7e5746bf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:01.918860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:01.919712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:01.920597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:01.920597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:01.920597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:01.920597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:01.922387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 267/529] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:01.927483 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158251c-6064-4ed8-8402-07c7e5746bf5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:01.940482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:02.030038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f982-f0a3-4a43-92f3-1925747fdaf7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:02.049674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158251c-6064-4ed8-8402-07c7e5746bf5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:02.069844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b5c1b-7213-4996-9436-2faf60dd05b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:02.070871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b5c1b-7213-4996-9436-2faf60dd05b1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:02.071184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b5c1b-7213-4996-9436-2faf60dd05b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:02.071491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b5c1b-7213-4996-9436-2faf60dd05b1 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:02.072404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b5c1b-7213-4996-9436-2faf60dd05b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:02.072715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b5c1b-7213-4996-9436-2faf60dd05b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:02.074206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b5c1b-7213-4996-9436-2faf60dd05b1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:02.088999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf5fda-3501-449a-b50f-2cd96389aa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:02.089658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf5fda-3501-449a-b50f-2cd96389aa93 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:02.089938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf5fda-3501-449a-b50f-2cd96389aa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:02.090314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf5fda-3501-449a-b50f-2cd96389aa93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:02.091152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf5fda-3501-449a-b50f-2cd96389aa93 None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=1b75a27126c9464989cec70e5104824c) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:02.091467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf5fda-3501-449a-b50f-2cd96389aa93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:02.093049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf5fda-3501-449a-b50f-2cd96389aa93 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:02.094444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 263/530] 10.222.0.69 () {56 vars in 1137 bytes} [Fri Jun 24 10:23:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:02.095413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b5c1b-7213-4996-9436-2faf60dd05b1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:02.104884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 268/531] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:01 2022] GET /identity/v3/role_assignments?user.id=1b75a27126c9464989cec70e5104824c&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:02.105869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf5fda-3501-449a-b50f-2cd96389aa93 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:02.126791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:02.195099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:02.209557 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b5c1b-7213-4996-9436-2faf60dd05b1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:02.249301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba32ae1-328c-4cde-98a2-8a765b0807c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:02.249992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba32ae1-328c-4cde-98a2-8a765b0807c0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:02.250279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba32ae1-328c-4cde-98a2-8a765b0807c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:02.250597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba32ae1-328c-4cde-98a2-8a765b0807c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:02.251351 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba32ae1-328c-4cde-98a2-8a765b0807c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:02.251662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba32ae1-328c-4cde-98a2-8a765b0807c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:02.253313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba32ae1-328c-4cde-98a2-8a765b0807c0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:02.266663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 264/532] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:02.272226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba32ae1-328c-4cde-98a2-8a765b0807c0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:02.358070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf5fda-3501-449a-b50f-2cd96389aa93 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:02.377309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba32ae1-328c-4cde-98a2-8a765b0807c0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:02.411835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1050dc9-f078-452a-9970-51e12972d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:02.412692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1050dc9-f078-452a-9970-51e12972d37d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:02.413028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1050dc9-f078-452a-9970-51e12972d37d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:02.413313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1050dc9-f078-452a-9970-51e12972d37d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:02.420457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1050dc9-f078-452a-9970-51e12972d37d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:02.421954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1050dc9-f078-452a-9970-51e12972d37d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:02.425143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:02.429417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e665c-657e-4173-97fb-9d092d8f80a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:02.430341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e665c-657e-4173-97fb-9d092d8f80a9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:02.430820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e665c-657e-4173-97fb-9d092d8f80a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:02.431289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e665c-657e-4173-97fb-9d092d8f80a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c1050dc9-f078-452a-9970-51e12972d37d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:02.438357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:02.439205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:02.440025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:02.440025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:02.440025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:02.440025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:02.440025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:02.440025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:02.440025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:02.440025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:02.440025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 265/533] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:02.445554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1050dc9-f078-452a-9970-51e12972d37d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:02.504922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e665c-657e-4173-97fb-9d092d8f80a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:02.506770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e665c-657e-4173-97fb-9d092d8f80a9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:02.525586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:02.636544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1050dc9-f078-452a-9970-51e12972d37d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:02.637503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 269/534] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:02.642796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e665c-657e-4173-97fb-9d092d8f80a9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:02.698821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796f71f-52f7-4c5f-a40f-fc14076e903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:02.699398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796f71f-52f7-4c5f-a40f-fc14076e903a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:02.699679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796f71f-52f7-4c5f-a40f-fc14076e903a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:02.699950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796f71f-52f7-4c5f-a40f-fc14076e903a None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:02.700834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796f71f-52f7-4c5f-a40f-fc14076e903a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:02.701143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796f71f-52f7-4c5f-a40f-fc14076e903a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:02.703442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796f71f-52f7-4c5f-a40f-fc14076e903a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:02.717256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 266/535] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:02.730025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796f71f-52f7-4c5f-a40f-fc14076e903a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:02.783868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e665c-657e-4173-97fb-9d092d8f80a9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:02.795659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:02.820239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e7dd9-99de-429a-a3af-c52d8c0a7fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:02.821731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7dd9-99de-429a-a3af-c52d8c0a7fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:02.822222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7dd9-99de-429a-a3af-c52d8c0a7fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:02.822720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7dd9-99de-429a-a3af-c52d8c0a7fd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:02.834348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e7dd9-99de-429a-a3af-c52d8c0a7fd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:02.836118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e7dd9-99de-429a-a3af-c52d8c0a7fd1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:02.841109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-fb6e7dd9-99de-429a-a3af-c52d8c0a7fd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:02.855142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:02.856013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:02.856856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:02.856856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:02.856856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:02.856856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:02.859146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 270/536] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:02.864227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e7dd9-99de-429a-a3af-c52d8c0a7fd1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:02.938964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796f71f-52f7-4c5f-a40f-fc14076e903a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:02.974451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e7dd9-99de-429a-a3af-c52d8c0a7fd1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:03.010862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dd743-d5bd-409a-9995-b1009d59418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:03.011865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dd743-d5bd-409a-9995-b1009d59418e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.012348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dd743-d5bd-409a-9995-b1009d59418e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.012883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dd743-d5bd-409a-9995-b1009d59418e None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.013843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dd743-d5bd-409a-9995-b1009d59418e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:03.014523 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dd743-d5bd-409a-9995-b1009d59418e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:03.016097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dd743-d5bd-409a-9995-b1009d59418e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:03.025427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a9f1d-726e-40b0-b26b-424f8a21d963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:03.026101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a9f1d-726e-40b0-b26b-424f8a21d963 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.026387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a9f1d-726e-40b0-b26b-424f8a21d963 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.026665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a9f1d-726e-40b0-b26b-424f8a21d963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.027507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a9f1d-726e-40b0-b26b-424f8a21d963 None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=a86de22b0e324c70aad792ec6b1a7e7d) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:03.027823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a9f1d-726e-40b0-b26b-424f8a21d963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:03.029280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a9f1d-726e-40b0-b26b-424f8a21d963 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:03.030220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 271/537] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:03.035128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dd743-d5bd-409a-9995-b1009d59418e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:03.040100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 267/538] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:02 2022] GET /identity/v3/role_assignments?user.id=a86de22b0e324c70aad792ec6b1a7e7d&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:03.064619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:03.149297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dd743-d5bd-409a-9995-b1009d59418e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:03.187083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa779312-9a3e-4fea-89b7-20badafe8ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:03.188388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa779312-9a3e-4fea-89b7-20badafe8ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.189106 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa779312-9a3e-4fea-89b7-20badafe8ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.189655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa779312-9a3e-4fea-89b7-20badafe8ef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.198164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa779312-9a3e-4fea-89b7-20badafe8ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:03.200102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa779312-9a3e-4fea-89b7-20badafe8ef1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-aa779312-9a3e-4fea-89b7-20badafe8ef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:03.217280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:03.218191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:03.219019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:03.219019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:03.219019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:03.219019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:03.219019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:03.219019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:03.219019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:03.219019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:03.221586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 272/539] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:03.227205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a9f1d-726e-40b0-b26b-424f8a21d963 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:03.241180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f12f13-c6a5-4f78-8db7-377dc1577a82 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.241787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f12f13-c6a5-4f78-8db7-377dc1577a82 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.242311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f12f13-c6a5-4f78-8db7-377dc1577a82 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.243895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 273/540] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:03.249765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5444d42-fe46-4952-8081-57aa9e8de508 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.250044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5444d42-fe46-4952-8081-57aa9e8de508 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.250334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5444d42-fe46-4952-8081-57aa9e8de508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.271199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e5444d42-fe46-4952-8081-57aa9e8de508 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:03.272515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e5444d42-fe46-4952-8081-57aa9e8de508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:03.342467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 274/541] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:03.366732 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5444d42-fe46-4952-8081-57aa9e8de508 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:03.397119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:03.419245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a9f1d-726e-40b0-b26b-424f8a21d963 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:03.462974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:03.491491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cebff-a399-48c7-88b9-483f196b40c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:03.492086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cebff-a399-48c7-88b9-483f196b40c2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.492373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cebff-a399-48c7-88b9-483f196b40c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.492649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cebff-a399-48c7-88b9-483f196b40c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.493483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cebff-a399-48c7-88b9-483f196b40c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:03.493794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cebff-a399-48c7-88b9-483f196b40c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:03.495245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cebff-a399-48c7-88b9-483f196b40c2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:03.519717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 268/542] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:03.525602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cebff-a399-48c7-88b9-483f196b40c2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:03.551067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5444d42-fe46-4952-8081-57aa9e8de508 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:03.613934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577545c-6c10-4862-8cb4-bae8ff7a4156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:03.614758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577545c-6c10-4862-8cb4-bae8ff7a4156 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.615041 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577545c-6c10-4862-8cb4-bae8ff7a4156 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.615318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577545c-6c10-4862-8cb4-bae8ff7a4156 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.632139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577545c-6c10-4862-8cb4-bae8ff7a4156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b577545c-6c10-4862-8cb4-bae8ff7a4156 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:03.640137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:03.641023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:03.641812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 275/543] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:03.647399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577545c-6c10-4862-8cb4-bae8ff7a4156 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:03.709763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cebff-a399-48c7-88b9-483f196b40c2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:03.739020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:03.767262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577545c-6c10-4862-8cb4-bae8ff7a4156 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:03.803027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d33da8-afcb-48fd-a5d0-ce8fb1b04e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:03.803864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d33da8-afcb-48fd-a5d0-ce8fb1b04e33 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.804250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d33da8-afcb-48fd-a5d0-ce8fb1b04e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.804662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d33da8-afcb-48fd-a5d0-ce8fb1b04e33 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.805588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d33da8-afcb-48fd-a5d0-ce8fb1b04e33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:03.805999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d33da8-afcb-48fd-a5d0-ce8fb1b04e33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:03.806517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf85348-c354-41e3-b6a8-b3676b8cf86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:03.807353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf85348-c354-41e3-b6a8-b3676b8cf86f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.807687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d33da8-afcb-48fd-a5d0-ce8fb1b04e33 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:03.807930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf85348-c354-41e3-b6a8-b3676b8cf86f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.808337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf85348-c354-41e3-b6a8-b3676b8cf86f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.813455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf85348-c354-41e3-b6a8-b3676b8cf86f None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=5586ef8925ef4cc496d358283c21c962) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:03.814008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf85348-c354-41e3-b6a8-b3676b8cf86f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:03.815636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf85348-c354-41e3-b6a8-b3676b8cf86f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:03.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 276/544] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:03.822988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d33da8-afcb-48fd-a5d0-ce8fb1b04e33 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:03.834833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 269/545] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:03 2022] GET /identity/v3/role_assignments?user.id=5586ef8925ef4cc496d358283c21c962&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:03.860524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea339b-6144-4937-bd52-910874ad31a0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.861111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea339b-6144-4937-bd52-910874ad31a0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.861462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea339b-6144-4937-bd52-910874ad31a0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.863252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 270/546] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:23:03.875286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fd0e3-10ad-4a7d-89eb-ee17c811e894 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.875794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fd0e3-10ad-4a7d-89eb-ee17c811e894 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.876177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fd0e3-10ad-4a7d-89eb-ee17c811e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.921965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c59fd0e3-10ad-4a7d-89eb-ee17c811e894 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:03.923208 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c59fd0e3-10ad-4a7d-89eb-ee17c811e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:03.942747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d33da8-afcb-48fd-a5d0-ce8fb1b04e33 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:03.980534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7b301-dfa1-4775-baa6-e209324da251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:03.981389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7b301-dfa1-4775-baa6-e209324da251 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:03.981670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7b301-dfa1-4775-baa6-e209324da251 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:03.981944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7b301-dfa1-4775-baa6-e209324da251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:03.985938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:03.994926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7b301-dfa1-4775-baa6-e209324da251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:03.996535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7b301-dfa1-4775-baa6-e209324da251 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:04.014726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-cfc7b301-dfa1-4775-baa6-e209324da251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:04.019339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:04.020492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:04.021537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:04.021537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:04.021537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:04.021537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:04.023409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 277/547] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:04.029017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7b301-dfa1-4775-baa6-e209324da251 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:04.074378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 271/548] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:04.114744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fd0e3-10ad-4a7d-89eb-ee17c811e894 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:04.214860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7b301-dfa1-4775-baa6-e209324da251 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:04.224795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fd0e3-10ad-4a7d-89eb-ee17c811e894 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:04.271136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e39ce-e478-4c20-87ec-4776af70f8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:04.271865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e39ce-e478-4c20-87ec-4776af70f8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:04.272146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e39ce-e478-4c20-87ec-4776af70f8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:04.272421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e39ce-e478-4c20-87ec-4776af70f8e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:04.291740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e39ce-e478-4c20-87ec-4776af70f8e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-261e39ce-e478-4c20-87ec-4776af70f8e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:04.299628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:04.300481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:04.301814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:04.303359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 272/549] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:04.308381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cefd0-f218-4c0a-9728-c4e2a5e77ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:04.309099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cefd0-f218-4c0a-9728-c4e2a5e77ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:04.309384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cefd0-f218-4c0a-9728-c4e2a5e77ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:04.309663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cefd0-f218-4c0a-9728-c4e2a5e77ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:04.310414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cefd0-f218-4c0a-9728-c4e2a5e77ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:04.310723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cefd0-f218-4c0a-9728-c4e2a5e77ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:04.312639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cefd0-f218-4c0a-9728-c4e2a5e77ad5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:04.315139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e39ce-e478-4c20-87ec-4776af70f8e6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:04.337943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 278/550] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:04.343488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cefd0-f218-4c0a-9728-c4e2a5e77ad5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:04.388789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:04.421611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:04.482777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cefd0-f218-4c0a-9728-c4e2a5e77ad5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:04.514295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e39ce-e478-4c20-87ec-4776af70f8e6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:04.520638 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18231b-9243-40fb-a250-5eb38add0d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:04.521761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18231b-9243-40fb-a250-5eb38add0d7f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:04.522269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18231b-9243-40fb-a250-5eb38add0d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:04.522733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18231b-9243-40fb-a250-5eb38add0d7f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:04.530680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18231b-9243-40fb-a250-5eb38add0d7f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:04.532467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18231b-9243-40fb-a250-5eb38add0d7f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-da18231b-9243-40fb-a250-5eb38add0d7f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:04.547645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:04.548554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:04.549534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:04.550733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 279/551] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:04.551600 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d0b90-6c55-49e1-808a-25ed71e311a7 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:04.551884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d0b90-6c55-49e1-808a-25ed71e311a7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:04.552160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d0b90-6c55-49e1-808a-25ed71e311a7 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:04.557584 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 280/552] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:04.558441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d0b90-6c55-49e1-808a-25ed71e311a7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:04.559635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421dc3e-bc55-4aed-8dbf-2cc3dfcb4e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:04.560595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421dc3e-bc55-4aed-8dbf-2cc3dfcb4e97 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:04.561465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421dc3e-bc55-4aed-8dbf-2cc3dfcb4e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:04.561975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421dc3e-bc55-4aed-8dbf-2cc3dfcb4e97 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:04.563936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421dc3e-bc55-4aed-8dbf-2cc3dfcb4e97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:04.564443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421dc3e-bc55-4aed-8dbf-2cc3dfcb4e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:04.566137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421dc3e-bc55-4aed-8dbf-2cc3dfcb4e97 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:04.578752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 273/553] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:04.581453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876280f-78ee-4821-8ad5-a50d5243b40a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:04.581962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876280f-78ee-4821-8ad5-a50d5243b40a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:04.582487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876280f-78ee-4821-8ad5-a50d5243b40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:04.605791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0876280f-78ee-4821-8ad5-a50d5243b40a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:04.607307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0876280f-78ee-4821-8ad5-a50d5243b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:04.689882 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d0b90-6c55-49e1-808a-25ed71e311a7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:04.696999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 274/554] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:04.697992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876280f-78ee-4821-8ad5-a50d5243b40a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:04.698954 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:04.741947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:04.744797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c38221-ae06-4da9-ab21-3162e079c042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:04.746267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c38221-ae06-4da9-ab21-3162e079c042 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:04.746762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c38221-ae06-4da9-ab21-3162e079c042 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:04.747223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c38221-ae06-4da9-ab21-3162e079c042 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:04.748501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c38221-ae06-4da9-ab21-3162e079c042 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:04.749034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c38221-ae06-4da9-ab21-3162e079c042 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:04.751435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c38221-ae06-4da9-ab21-3162e079c042 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:04.773533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 281/555] 10.222.0.69 () {56 vars in 1135 bytes} [Fri Jun 24 10:23:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:04.774719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c38221-ae06-4da9-ab21-3162e079c042 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:04.828572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876280f-78ee-4821-8ad5-a50d5243b40a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:04.870795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90066833-290b-4e6b-98d9-3acb03c23b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:04.871784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90066833-290b-4e6b-98d9-3acb03c23b89 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:04.872135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90066833-290b-4e6b-98d9-3acb03c23b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:04.872475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90066833-290b-4e6b-98d9-3acb03c23b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:04.883810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90066833-290b-4e6b-98d9-3acb03c23b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:04.885402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90066833-290b-4e6b-98d9-3acb03c23b89 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:04.893640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c38221-ae06-4da9-ab21-3162e079c042 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-90066833-290b-4e6b-98d9-3acb03c23b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:04.904812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:04.905671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:04.906496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:04.906496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:04.906496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:04.906496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:04.906950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 275/556] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:04.907955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90066833-290b-4e6b-98d9-3acb03c23b89 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:04.937203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc323c-c8c8-4afe-aa4c-ef4b03a9c8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:04.938884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc323c-c8c8-4afe-aa4c-ef4b03a9c8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:04.939306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc323c-c8c8-4afe-aa4c-ef4b03a9c8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:04.939947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc323c-c8c8-4afe-aa4c-ef4b03a9c8c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:04.953486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc323c-c8c8-4afe-aa4c-ef4b03a9c8c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f0fc323c-c8c8-4afe-aa4c-ef4b03a9c8c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:04.964300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:04.965479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:04.966944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:04.969992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 282/557] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:04.970984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc323c-c8c8-4afe-aa4c-ef4b03a9c8c7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:04.997157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:05.006406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:05.028163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90066833-290b-4e6b-98d9-3acb03c23b89 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:05.064509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fda30-44b7-4825-83e1-74d44e5a6def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:05.065323 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fda30-44b7-4825-83e1-74d44e5a6def None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.065706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fda30-44b7-4825-83e1-74d44e5a6def None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.066097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fda30-44b7-4825-83e1-74d44e5a6def None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.078828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc323c-c8c8-4afe-aa4c-ef4b03a9c8c7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:05.115056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce610db-8e9e-401b-8747-cf1bfe930581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:05.116028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce610db-8e9e-401b-8747-cf1bfe930581 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.116482 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce610db-8e9e-401b-8747-cf1bfe930581 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.116967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce610db-8e9e-401b-8747-cf1bfe930581 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.117893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce610db-8e9e-401b-8747-cf1bfe930581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:05.118367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce610db-8e9e-401b-8747-cf1bfe930581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:05.119906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce610db-8e9e-401b-8747-cf1bfe930581 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:05.133093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 283/558] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:05.134267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce610db-8e9e-401b-8747-cf1bfe930581 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:05.141070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fda30-44b7-4825-83e1-74d44e5a6def None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:05.142708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fda30-44b7-4825-83e1-74d44e5a6def None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:05.222059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:05.226902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:05.239698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce610db-8e9e-401b-8747-cf1bfe930581 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:05.253654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 276/559] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:05.254677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fda30-44b7-4825-83e1-74d44e5a6def None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:05.275552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f59e53-bd74-49df-824a-dfb3ff14471e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:05.276506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f59e53-bd74-49df-824a-dfb3ff14471e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.276984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f59e53-bd74-49df-824a-dfb3ff14471e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.277458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f59e53-bd74-49df-824a-dfb3ff14471e None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.278558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f59e53-bd74-49df-824a-dfb3ff14471e None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:05.279127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f59e53-bd74-49df-824a-dfb3ff14471e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:05.280861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f59e53-bd74-49df-824a-dfb3ff14471e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:05.290185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 284/560] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:05.291353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f59e53-bd74-49df-824a-dfb3ff14471e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:05.371377 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fda30-44b7-4825-83e1-74d44e5a6def None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:05.400271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f59e53-bd74-49df-824a-dfb3ff14471e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:05.407387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f899-299c-4d1f-a743-c79a19bdc4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:05.408231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f899-299c-4d1f-a743-c79a19bdc4cc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.408552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f899-299c-4d1f-a743-c79a19bdc4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.408857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f899-299c-4d1f-a743-c79a19bdc4cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.416595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5f899-299c-4d1f-a743-c79a19bdc4cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:05.418329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5f899-299c-4d1f-a743-c79a19bdc4cc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4fe5f899-299c-4d1f-a743-c79a19bdc4cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:05.434563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:05.435554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 277/561] 10.222.0.69 () {56 vars in 1093 bytes} [Fri Jun 24 10:23:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:05.436895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f899-299c-4d1f-a743-c79a19bdc4cc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:05.445728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f0402-9a8a-4699-8e6a-179759a0444a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:05.446552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f0402-9a8a-4699-8e6a-179759a0444a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.446829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f0402-9a8a-4699-8e6a-179759a0444a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.447111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f0402-9a8a-4699-8e6a-179759a0444a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.461171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f0402-9a8a-4699-8e6a-179759a0444a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:05.462714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f0402-9a8a-4699-8e6a-179759a0444a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:05.469844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-be7f0402-9a8a-4699-8e6a-179759a0444a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:05.482255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:05.483509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:05.484663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:05.484663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:05.484663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:05.484663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:05.490637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:05.491108 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 285/562] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:05.492905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012272a0-bd7f-467d-8c0c-5f7fc957d3ef None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.493225 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012272a0-bd7f-467d-8c0c-5f7fc957d3ef None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.493503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012272a0-bd7f-467d-8c0c-5f7fc957d3ef None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.495237 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 286/563] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:05.495861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012272a0-bd7f-467d-8c0c-5f7fc957d3ef None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:05.586832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f899-299c-4d1f-a743-c79a19bdc4cc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:05.615383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012272a0-bd7f-467d-8c0c-5f7fc957d3ef None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:05.634034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9adf9-38ae-4c73-b9ae-a4a391d37774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:05.634820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9adf9-38ae-4c73-b9ae-a4a391d37774 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.635090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9adf9-38ae-4c73-b9ae-a4a391d37774 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.635365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9adf9-38ae-4c73-b9ae-a4a391d37774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.651286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9adf9-38ae-4c73-b9ae-a4a391d37774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:05.652770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9adf9-38ae-4c73-b9ae-a4a391d37774 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:05.658547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c21076-397b-42e7-bbbf-9a90be1ef67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:05.659229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c21076-397b-42e7-bbbf-9a90be1ef67f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.659512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c21076-397b-42e7-bbbf-9a90be1ef67f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.659790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c21076-397b-42e7-bbbf-9a90be1ef67f None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.660627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c21076-397b-42e7-bbbf-9a90be1ef67f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:05.661266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c21076-397b-42e7-bbbf-9a90be1ef67f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:05.663176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c21076-397b-42e7-bbbf-9a90be1ef67f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6cb9adf9-38ae-4c73-b9ae-a4a391d37774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:05.678175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:05.679093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:05.679953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:05.679953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:05.679953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:05.679953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:05.681770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 278/564] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:05.682623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9adf9-38ae-4c73-b9ae-a4a391d37774 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:05.690543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 287/565] 10.222.0.69 () {56 vars in 1133 bytes} [Fri Jun 24 10:23:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:05.692395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a588e-9e63-4186-ab19-f02ba6f56ff0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.692845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a588e-9e63-4186-ab19-f02ba6f56ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.693377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a588e-9e63-4186-ab19-f02ba6f56ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.714029 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4b5a588e-9e63-4186-ab19-f02ba6f56ff0 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:05.715231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4b5a588e-9e63-4186-ab19-f02ba6f56ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:05.775813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:05.804282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 288/566] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:05.806427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a588e-9e63-4186-ab19-f02ba6f56ff0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:05.813583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:05.821439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9adf9-38ae-4c73-b9ae-a4a391d37774 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:05.875424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83dc22-00cb-4cd7-a873-3f166bc72653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:05.876016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83dc22-00cb-4cd7-a873-3f166bc72653 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.876294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83dc22-00cb-4cd7-a873-3f166bc72653 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.876564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83dc22-00cb-4cd7-a873-3f166bc72653 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:05.877331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83dc22-00cb-4cd7-a873-3f166bc72653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:05.877631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83dc22-00cb-4cd7-a873-3f166bc72653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:05.878999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83dc22-00cb-4cd7-a873-3f166bc72653 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:05.897749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 279/567] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:05.898441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83dc22-00cb-4cd7-a873-3f166bc72653 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:05.942538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a588e-9e63-4186-ab19-f02ba6f56ff0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:05.997464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ffb6d-4941-47d9-8405-ef14bff7dd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:05.998461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ffb6d-4941-47d9-8405-ef14bff7dd6b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:05.999147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ffb6d-4941-47d9-8405-ef14bff7dd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:05.999438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ffb6d-4941-47d9-8405-ef14bff7dd6b None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.000178 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ffb6d-4941-47d9-8405-ef14bff7dd6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:06.000493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ffb6d-4941-47d9-8405-ef14bff7dd6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.002043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ffb6d-4941-47d9-8405-ef14bff7dd6b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.018285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 289/568] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:06.018285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ffb6d-4941-47d9-8405-ef14bff7dd6b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.025554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83dc22-00cb-4cd7-a873-3f166bc72653 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.071358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9d65c-20a8-4c28-9b90-9421c73a89c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.072183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9d65c-20a8-4c28-9b90-9421c73a89c8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.072469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9d65c-20a8-4c28-9b90-9421c73a89c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.072767 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9d65c-20a8-4c28-9b90-9421c73a89c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.072857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:06.090615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9d65c-20a8-4c28-9b90-9421c73a89c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.093322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9d65c-20a8-4c28-9b90-9421c73a89c8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-12f9d65c-20a8-4c28-9b90-9421c73a89c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:06.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:06.113869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:06.114745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:06.114745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:06.114745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:06.114745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:06.114745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 280/569] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:06.114745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9d65c-20a8-4c28-9b90-9421c73a89c8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.119130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:06.143200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ffb6d-4941-47d9-8405-ef14bff7dd6b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.181444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b7ff0-1e8c-491d-9f31-b29e50eaa1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.182236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b7ff0-1e8c-491d-9f31-b29e50eaa1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.182500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b7ff0-1e8c-491d-9f31-b29e50eaa1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.182800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b7ff0-1e8c-491d-9f31-b29e50eaa1e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.194677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b7ff0-1e8c-491d-9f31-b29e50eaa1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-6a6b7ff0-1e8c-491d-9f31-b29e50eaa1e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:06.207187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:06.208164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:06.209019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 290/570] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:06.209705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b7ff0-1e8c-491d-9f31-b29e50eaa1e6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.236023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9d65c-20a8-4c28-9b90-9421c73a89c8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.272921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1b2f1-0cd9-425a-aa47-11da178ca534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.273725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1b2f1-0cd9-425a-aa47-11da178ca534 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.274005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1b2f1-0cd9-425a-aa47-11da178ca534 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.274278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1b2f1-0cd9-425a-aa47-11da178ca534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.283871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1b2f1-0cd9-425a-aa47-11da178ca534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.285365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1b2f1-0cd9-425a-aa47-11da178ca534 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c3f1b2f1-0cd9-425a-aa47-11da178ca534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:06.302346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:06.304152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 281/571] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc72642-b5da-40a6-af56-6ac30d8c088f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc72642-b5da-40a6-af56-6ac30d8c088f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.306769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc72642-b5da-40a6-af56-6ac30d8c088f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.306769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 282/572] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:06.307523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc72642-b5da-40a6-af56-6ac30d8c088f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.331055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b7ff0-1e8c-491d-9f31-b29e50eaa1e6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.338394 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:06.366120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48534005-4aea-40ed-a468-20846f9f384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.366909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48534005-4aea-40ed-a468-20846f9f384a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.367188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48534005-4aea-40ed-a468-20846f9f384a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.367455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48534005-4aea-40ed-a468-20846f9f384a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.374280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:06.375177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48534005-4aea-40ed-a468-20846f9f384a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.376706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48534005-4aea-40ed-a468-20846f9f384a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-48534005-4aea-40ed-a468-20846f9f384a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:06.391601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:06.392565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:06.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 291/573] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:06.394159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48534005-4aea-40ed-a468-20846f9f384a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.417040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc72642-b5da-40a6-af56-6ac30d8c088f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.453560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42794a-18b2-42fb-b674-bf94f3d6c69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.454373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42794a-18b2-42fb-b674-bf94f3d6c69a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.454655 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42794a-18b2-42fb-b674-bf94f3d6c69a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.455237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42794a-18b2-42fb-b674-bf94f3d6c69a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.455979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42794a-18b2-42fb-b674-bf94f3d6c69a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:06.456267 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42794a-18b2-42fb-b674-bf94f3d6c69a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.457722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42794a-18b2-42fb-b674-bf94f3d6c69a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.469863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 283/574] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:06.470649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42794a-18b2-42fb-b674-bf94f3d6c69a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.503976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48534005-4aea-40ed-a468-20846f9f384a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.538952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f0fa2-9a7d-484a-b1af-b5dd85d0c4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.539628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f0fa2-9a7d-484a-b1af-b5dd85d0c4fc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.539941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f0fa2-9a7d-484a-b1af-b5dd85d0c4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.540191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f0fa2-9a7d-484a-b1af-b5dd85d0c4fc None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.540976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f0fa2-9a7d-484a-b1af-b5dd85d0c4fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:06.541290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f0fa2-9a7d-484a-b1af-b5dd85d0c4fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.542667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f0fa2-9a7d-484a-b1af-b5dd85d0c4fc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.550998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 292/575] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:06.552184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38aa5c9-ccfe-4879-a243-9597ac2728b6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.552477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38aa5c9-ccfe-4879-a243-9597ac2728b6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.552756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38aa5c9-ccfe-4879-a243-9597ac2728b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.568595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c38aa5c9-ccfe-4879-a243-9597ac2728b6 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:06.568786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42794a-18b2-42fb-b674-bf94f3d6c69a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.569793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c38aa5c9-ccfe-4879-a243-9597ac2728b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:06.593313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:06.593496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:06.604500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949e0a1-1c1b-448a-8096-20d9d4f5c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.605209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949e0a1-1c1b-448a-8096-20d9d4f5c694 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.605489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949e0a1-1c1b-448a-8096-20d9d4f5c694 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.605759 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949e0a1-1c1b-448a-8096-20d9d4f5c694 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.606546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949e0a1-1c1b-448a-8096-20d9d4f5c694 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:06.606851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949e0a1-1c1b-448a-8096-20d9d4f5c694 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.608655 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949e0a1-1c1b-448a-8096-20d9d4f5c694 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.626279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 284/576] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:06.626279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949e0a1-1c1b-448a-8096-20d9d4f5c694 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.639716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 293/577] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:06.640574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38aa5c9-ccfe-4879-a243-9597ac2728b6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.731609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949e0a1-1c1b-448a-8096-20d9d4f5c694 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.747924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38aa5c9-ccfe-4879-a243-9597ac2728b6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.766714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c8bfd-c30d-4f3f-80d2-c911caff442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.767370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c8bfd-c30d-4f3f-80d2-c911caff442a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.767646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c8bfd-c30d-4f3f-80d2-c911caff442a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.767916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c8bfd-c30d-4f3f-80d2-c911caff442a None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.768763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c8bfd-c30d-4f3f-80d2-c911caff442a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:06.769078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c8bfd-c30d-4f3f-80d2-c911caff442a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.770502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c8bfd-c30d-4f3f-80d2-c911caff442a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.780904 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf5e8c-a59b-4212-a409-d1e42eec469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.781705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf5e8c-a59b-4212-a409-d1e42eec469b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.781988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf5e8c-a59b-4212-a409-d1e42eec469b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.782260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf5e8c-a59b-4212-a409-d1e42eec469b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.787366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 285/578] 10.222.0.69 () {56 vars in 1137 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:06.788169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c8bfd-c30d-4f3f-80d2-c911caff442a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.789109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf5e8c-a59b-4212-a409-d1e42eec469b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.790562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf5e8c-a59b-4212-a409-d1e42eec469b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-5caf5e8c-a59b-4212-a409-d1e42eec469b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:06.805002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:06.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:06.806832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:06.806832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:06.806832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:06.806832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:06.806832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:06.806832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:06.806832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:06.806832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:06.806832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 294/579] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:06.807302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf5e8c-a59b-4212-a409-d1e42eec469b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.826033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:06.836349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:06.900619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c8bfd-c30d-4f3f-80d2-c911caff442a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.915583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf5e8c-a59b-4212-a409-d1e42eec469b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:06.938022 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581007b0-a254-4c5c-b55d-735f5469ce54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.938829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581007b0-a254-4c5c-b55d-735f5469ce54 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.939107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581007b0-a254-4c5c-b55d-735f5469ce54 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.939375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581007b0-a254-4c5c-b55d-735f5469ce54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.948685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd5e87-ba91-4afd-b8d2-ca72a7226702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:06.949360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd5e87-ba91-4afd-b8d2-ca72a7226702 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:06.949526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581007b0-a254-4c5c-b55d-735f5469ce54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.949626 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd5e87-ba91-4afd-b8d2-ca72a7226702 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:06.949902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd5e87-ba91-4afd-b8d2-ca72a7226702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:06.950728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd5e87-ba91-4afd-b8d2-ca72a7226702 None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=44d5bd3f9dfe45ecb8d1a51d248f3c54) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:06.950969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581007b0-a254-4c5c-b55d-735f5469ce54 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.951069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd5e87-ba91-4afd-b8d2-ca72a7226702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:06.952494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd5e87-ba91-4afd-b8d2-ca72a7226702 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:06.958390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 295/580] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/role_assignments?user.id=44d5bd3f9dfe45ecb8d1a51d248f3c54&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:06.959160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd5e87-ba91-4afd-b8d2-ca72a7226702 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-581007b0-a254-4c5c-b55d-735f5469ce54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:06.969218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:06.970416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:06.971475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:06.971475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:06.971475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:06.971475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:06.973512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 286/581] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:06.975062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581007b0-a254-4c5c-b55d-735f5469ce54 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.063064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:07.083660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd5e87-ba91-4afd-b8d2-ca72a7226702 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.084408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:07.093766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581007b0-a254-4c5c-b55d-735f5469ce54 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.119124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be735a3-7c18-4007-8290-ab984a3f2f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.119930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be735a3-7c18-4007-8290-ab984a3f2f5c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.120213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be735a3-7c18-4007-8290-ab984a3f2f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.120489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be735a3-7c18-4007-8290-ab984a3f2f5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.131600 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be735a3-7c18-4007-8290-ab984a3f2f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:07.136646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30297fe-84ef-4c19-a58c-69d23ea68fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.137478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30297fe-84ef-4c19-a58c-69d23ea68fab None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.137760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30297fe-84ef-4c19-a58c-69d23ea68fab None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.138030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30297fe-84ef-4c19-a58c-69d23ea68fab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0be735a3-7c18-4007-8290-ab984a3f2f5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:07.140011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:07.140898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:07.141707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 296/582] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:07.142402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be735a3-7c18-4007-8290-ab984a3f2f5c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.153933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30297fe-84ef-4c19-a58c-69d23ea68fab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:07.155413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30297fe-84ef-4c19-a58c-69d23ea68fab None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f30297fe-84ef-4c19-a58c-69d23ea68fab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:07.172648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:07.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:07.174809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:07.174809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:07.174809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:07.174809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:07.174809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 287/583] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:07.174809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30297fe-84ef-4c19-a58c-69d23ea68fab None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.255932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be735a3-7c18-4007-8290-ab984a3f2f5c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.278316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30297fe-84ef-4c19-a58c-69d23ea68fab None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.290564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2e13-c1a3-4ea2-ae56-2abdb56bd4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.291331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2e13-c1a3-4ea2-ae56-2abdb56bd4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.291614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2e13-c1a3-4ea2-ae56-2abdb56bd4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.291888 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2e13-c1a3-4ea2-ae56-2abdb56bd4f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.292669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb2e13-c1a3-4ea2-ae56-2abdb56bd4f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:07.293006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb2e13-c1a3-4ea2-ae56-2abdb56bd4f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:07.294425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb2e13-c1a3-4ea2-ae56-2abdb56bd4f7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:07.312457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 297/584] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:07.312457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2e13-c1a3-4ea2-ae56-2abdb56bd4f7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.312457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:07.319668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5dd40-996c-4fd8-849f-c8cea76fa1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.320321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5dd40-996c-4fd8-849f-c8cea76fa1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.320603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5dd40-996c-4fd8-849f-c8cea76fa1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.320906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5dd40-996c-4fd8-849f-c8cea76fa1b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.321754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5dd40-996c-4fd8-849f-c8cea76fa1b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:07.322062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5dd40-996c-4fd8-849f-c8cea76fa1b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:07.323423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5dd40-996c-4fd8-849f-c8cea76fa1b7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:07.340192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 288/585] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:07.340192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5dd40-996c-4fd8-849f-c8cea76fa1b7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.348020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:07.417197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2e13-c1a3-4ea2-ae56-2abdb56bd4f7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.444456 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5dd40-996c-4fd8-849f-c8cea76fa1b7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.460087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d887bf-feab-4e84-a17c-b63addfe1c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.460786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d887bf-feab-4e84-a17c-b63addfe1c01 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.461072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d887bf-feab-4e84-a17c-b63addfe1c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.461354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d887bf-feab-4e84-a17c-b63addfe1c01 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.462130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d887bf-feab-4e84-a17c-b63addfe1c01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:07.462424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d887bf-feab-4e84-a17c-b63addfe1c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:07.463793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d887bf-feab-4e84-a17c-b63addfe1c01 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:07.472388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 298/586] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:07.473219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d887bf-feab-4e84-a17c-b63addfe1c01 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.488973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd42d29-6f03-4cfd-a2ba-d0a65a3b90f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.489954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd42d29-6f03-4cfd-a2ba-d0a65a3b90f5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.490461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd42d29-6f03-4cfd-a2ba-d0a65a3b90f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.490792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd42d29-6f03-4cfd-a2ba-d0a65a3b90f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.491626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd42d29-6f03-4cfd-a2ba-d0a65a3b90f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:07.491957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd42d29-6f03-4cfd-a2ba-d0a65a3b90f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:07.493439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd42d29-6f03-4cfd-a2ba-d0a65a3b90f5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:07.506003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 289/587] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:07.506803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd42d29-6f03-4cfd-a2ba-d0a65a3b90f5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.545792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:07.577757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:07.586824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d887bf-feab-4e84-a17c-b63addfe1c01 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.615834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd42d29-6f03-4cfd-a2ba-d0a65a3b90f5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.619266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3b20b-37d6-41a0-a0d1-2be66cfb4f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.620160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3b20b-37d6-41a0-a0d1-2be66cfb4f82 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.620438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3b20b-37d6-41a0-a0d1-2be66cfb4f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.620709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3b20b-37d6-41a0-a0d1-2be66cfb4f82 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/bec9fee282a64933aa0f6fbdd91a93c4/roles/6b7e9f2fcda040b18213464589952d67` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.635392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3b20b-37d6-41a0-a0d1-2be66cfb4f82 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=bec9fee282a64933aa0f6fbdd91a93c4, role_id=6b7e9f2fcda040b18213464589952d67)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:07.636873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3b20b-37d6-41a0-a0d1-2be66cfb4f82 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:07.650229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 299/588] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:23:07 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/bec9fee282a64933aa0f6fbdd91a93c4/roles/6b7e9f2fcda040b18213464589952d67 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:07.652924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf48a9f-d88b-4c9c-aa97-72422414b322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.653746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf48a9f-d88b-4c9c-aa97-72422414b322 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.654030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf48a9f-d88b-4c9c-aa97-72422414b322 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.654310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf48a9f-d88b-4c9c-aa97-72422414b322 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.654616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3b20b-37d6-41a0-a0d1-2be66cfb4f82 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.663256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf48a9f-d88b-4c9c-aa97-72422414b322 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:07.664831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf48a9f-d88b-4c9c-aa97-72422414b322 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-cdf48a9f-d88b-4c9c-aa97-72422414b322 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:07.682057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:07.683114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:07.684027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 290/589] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:07.685153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf48a9f-d88b-4c9c-aa97-72422414b322 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.805410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3b20b-37d6-41a0-a0d1-2be66cfb4f82 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.807877 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf48a9f-d88b-4c9c-aa97-72422414b322 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:07.825966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:07.840702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:07.850074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba14f-cc0b-4896-a1f5-f69b472d3e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.850878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba14f-cc0b-4896-a1f5-f69b472d3e49 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.851158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba14f-cc0b-4896-a1f5-f69b472d3e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.851430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba14f-cc0b-4896-a1f5-f69b472d3e49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.858511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc72a19-9d19-43b0-9bea-1be576ddd385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:07.858778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ba14f-cc0b-4896-a1f5-f69b472d3e49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:07.859314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc72a19-9d19-43b0-9bea-1be576ddd385 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:07.859596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc72a19-9d19-43b0-9bea-1be576ddd385 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:07.859869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc72a19-9d19-43b0-9bea-1be576ddd385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:07.860226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ba14f-cc0b-4896-a1f5-f69b472d3e49 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:07.874932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc72a19-9d19-43b0-9bea-1be576ddd385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:07.874932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc72a19-9d19-43b0-9bea-1be576ddd385 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-9e5ba14f-cc0b-4896-a1f5-f69b472d3e49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:07.875242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:07.876217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 291/590] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:07.877232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba14f-cc0b-4896-a1f5-f69b472d3e49 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3cc72a19-9d19-43b0-9bea-1be576ddd385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:07.900332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:07.901339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:07.902207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:07.902207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:07.902207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:07.902207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:07.902207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 300/591] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:07.902928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc72a19-9d19-43b0-9bea-1be576ddd385 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.001553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba14f-cc0b-4896-a1f5-f69b472d3e49 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.019843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc72a19-9d19-43b0-9bea-1be576ddd385 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.038530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad8c44-ba05-4f2e-a34d-36bdc33f53c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.039205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad8c44-ba05-4f2e-a34d-36bdc33f53c0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.039490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad8c44-ba05-4f2e-a34d-36bdc33f53c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.039765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad8c44-ba05-4f2e-a34d-36bdc33f53c0 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.040619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad8c44-ba05-4f2e-a34d-36bdc33f53c0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:08.040954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad8c44-ba05-4f2e-a34d-36bdc33f53c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.042717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad8c44-ba05-4f2e-a34d-36bdc33f53c0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.068161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 292/592] 10.222.0.69 () {56 vars in 1139 bytes} [Fri Jun 24 10:23:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:08.068161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad8c44-ba05-4f2e-a34d-36bdc33f53c0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.073833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936527c-dc38-496b-a807-9536670c8640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.073833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936527c-dc38-496b-a807-9536670c8640 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.074189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936527c-dc38-496b-a807-9536670c8640 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.074413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936527c-dc38-496b-a807-9536670c8640 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.075250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936527c-dc38-496b-a807-9536670c8640 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:08.075563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936527c-dc38-496b-a807-9536670c8640 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.077070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936527c-dc38-496b-a807-9536670c8640 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.090142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 301/593] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:08.091093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936527c-dc38-496b-a807-9536670c8640 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.119079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:08.127963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:08.197676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad8c44-ba05-4f2e-a34d-36bdc33f53c0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.199650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936527c-dc38-496b-a807-9536670c8640 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.237986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74cd9a-86b5-4192-aad8-6a1fcf934252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.239019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e61521-5a49-4b72-9f1f-d79bda84dd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.239305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74cd9a-86b5-4192-aad8-6a1fcf934252 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.239796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74cd9a-86b5-4192-aad8-6a1fcf934252 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.239898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61521-5a49-4b72-9f1f-d79bda84dd43 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.240093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61521-5a49-4b72-9f1f-d79bda84dd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.240372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61521-5a49-4b72-9f1f-d79bda84dd43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.240728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74cd9a-86b5-4192-aad8-6a1fcf934252 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.241843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74cd9a-86b5-4192-aad8-6a1fcf934252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:08.242587 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74cd9a-86b5-4192-aad8-6a1fcf934252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.244566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74cd9a-86b5-4192-aad8-6a1fcf934252 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.251834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e61521-5a49-4b72-9f1f-d79bda84dd43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.253318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e61521-5a49-4b72-9f1f-d79bda84dd43 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.259271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 293/594] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:08.260837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74cd9a-86b5-4192-aad8-6a1fcf934252 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b1e61521-5a49-4b72-9f1f-d79bda84dd43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:08.274660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:08.275544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:08.276395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:08.276395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:08.276395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:08.276395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:08.280671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 302/595] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:08.281727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e61521-5a49-4b72-9f1f-d79bda84dd43 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.366397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:08.390037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e61521-5a49-4b72-9f1f-d79bda84dd43 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.395431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74cd9a-86b5-4192-aad8-6a1fcf934252 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.398519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:08.431604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f430a6-acc1-412c-b5e8-c573427f6228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.432504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f430a6-acc1-412c-b5e8-c573427f6228 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.432813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f430a6-acc1-412c-b5e8-c573427f6228 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.433096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f430a6-acc1-412c-b5e8-c573427f6228 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/1b75a27126c9464989cec70e5104824c/roles/6b7e9f2fcda040b18213464589952d67` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.434436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d8f49-13ea-49fe-a9eb-952abc595f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.435614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d8f49-13ea-49fe-a9eb-952abc595f2d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.436198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d8f49-13ea-49fe-a9eb-952abc595f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.437137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d8f49-13ea-49fe-a9eb-952abc595f2d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.449731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f430a6-acc1-412c-b5e8-c573427f6228 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=1b75a27126c9464989cec70e5104824c, role_id=6b7e9f2fcda040b18213464589952d67)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.451190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d8f49-13ea-49fe-a9eb-952abc595f2d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.452050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f430a6-acc1-412c-b5e8-c573427f6228 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.452833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d8f49-13ea-49fe-a9eb-952abc595f2d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.469404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 294/596] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:23:08 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/1b75a27126c9464989cec70e5104824c/roles/6b7e9f2fcda040b18213464589952d67 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:08.470496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f430a6-acc1-412c-b5e8-c573427f6228 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-c00d8f49-13ea-49fe-a9eb-952abc595f2d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:08.476975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:08.478148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:23:08.479277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:08.486046 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 303/597] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:08.487599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d8f49-13ea-49fe-a9eb-952abc595f2d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.602122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d8f49-13ea-49fe-a9eb-952abc595f2d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.609567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f430a6-acc1-412c-b5e8-c573427f6228 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.639722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d56529-a1d0-41ba-8a83-f23e82b89a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.640425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d56529-a1d0-41ba-8a83-f23e82b89a6c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.640894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d56529-a1d0-41ba-8a83-f23e82b89a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.641320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d56529-a1d0-41ba-8a83-f23e82b89a6c None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.642312 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d56529-a1d0-41ba-8a83-f23e82b89a6c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:08.642734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d56529-a1d0-41ba-8a83-f23e82b89a6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.644328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d56529-a1d0-41ba-8a83-f23e82b89a6c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.654690 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:08.659369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edd119-85f3-4d6b-ab62-042e697d1bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.660278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edd119-85f3-4d6b-ab62-042e697d1bad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.660729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edd119-85f3-4d6b-ab62-042e697d1bad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.661171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edd119-85f3-4d6b-ab62-042e697d1bad None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.663314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edd119-85f3-4d6b-ab62-042e697d1bad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:08.663783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edd119-85f3-4d6b-ab62-042e697d1bad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.665625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edd119-85f3-4d6b-ab62-042e697d1bad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.671810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 304/598] 10.222.0.69 () {56 vars in 1137 bytes} [Fri Jun 24 10:23:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:08.673232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d56529-a1d0-41ba-8a83-f23e82b89a6c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.680951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 295/599] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:08.688910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edd119-85f3-4d6b-ab62-042e697d1bad None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.706775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:08.796527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d56529-a1d0-41ba-8a83-f23e82b89a6c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.801624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edd119-85f3-4d6b-ab62-042e697d1bad None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:08.837368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13891a59-efd5-4592-9e54-3332d9c12916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.839105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13891a59-efd5-4592-9e54-3332d9c12916 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.839536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13891a59-efd5-4592-9e54-3332d9c12916 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.840146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13891a59-efd5-4592-9e54-3332d9c12916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.841438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b41dc-5258-4bb3-8dd7-f98b8954a221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:08.842550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b41dc-5258-4bb3-8dd7-f98b8954a221 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:08.842935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b41dc-5258-4bb3-8dd7-f98b8954a221 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:08.843311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b41dc-5258-4bb3-8dd7-f98b8954a221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:08.851230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b41dc-5258-4bb3-8dd7-f98b8954a221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.852865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b41dc-5258-4bb3-8dd7-f98b8954a221 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.854209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13891a59-efd5-4592-9e54-3332d9c12916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:08.856020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13891a59-efd5-4592-9e54-3332d9c12916 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-176b41dc-5258-4bb3-8dd7-f98b8954a221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:08.868644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:08.869622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:08.870472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:08.870472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:08.870472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:08.870472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:08.870472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:08.870472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:08.870472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:08.870472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:08.870472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 296/600] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:08.881700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b41dc-5258-4bb3-8dd7-f98b8954a221 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-13891a59-efd5-4592-9e54-3332d9c12916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:08.881843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:08.882628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:08.883471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:08.883471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:08.883471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:08.883471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:08.887060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 305/601] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:08.890153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13891a59-efd5-4592-9e54-3332d9c12916 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:08.987208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:09.036142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:09.084880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b41dc-5258-4bb3-8dd7-f98b8954a221 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:09.089649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13891a59-efd5-4592-9e54-3332d9c12916 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:09.133038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06834a28-9d58-4726-b3f7-c300d65cef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:09.134252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06834a28-9d58-4726-b3f7-c300d65cef1f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.134743 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06834a28-9d58-4726-b3f7-c300d65cef1f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.135222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06834a28-9d58-4726-b3f7-c300d65cef1f None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.136139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06834a28-9d58-4726-b3f7-c300d65cef1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:09.136646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06834a28-9d58-4726-b3f7-c300d65cef1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:09.138582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06834a28-9d58-4726-b3f7-c300d65cef1f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:09.151618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 306/602] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:09.156982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06834a28-9d58-4726-b3f7-c300d65cef1f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:09.175910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f2d84-4511-4661-a04c-948117f53f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:09.176587 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f2d84-4511-4661-a04c-948117f53f67 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.176907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f2d84-4511-4661-a04c-948117f53f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.177245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f2d84-4511-4661-a04c-948117f53f67 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.177979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f2d84-4511-4661-a04c-948117f53f67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:09.178289 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f2d84-4511-4661-a04c-948117f53f67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:09.179930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f2d84-4511-4661-a04c-948117f53f67 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:09.193984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 297/603] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:09.199419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1fc93-2df1-4149-bd9b-6e569d1ac3e7 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.199419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1fc93-2df1-4149-bd9b-6e569d1ac3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.199419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1fc93-2df1-4149-bd9b-6e569d1ac3e7 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.199419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 298/604] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:09.201920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1fc93-2df1-4149-bd9b-6e569d1ac3e7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:09.294140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06834a28-9d58-4726-b3f7-c300d65cef1f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:09.304476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:09.317341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1fc93-2df1-4149-bd9b-6e569d1ac3e7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:09.333611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6105954-5d36-4f8e-bf7c-cdda5ef25332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:09.334865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6105954-5d36-4f8e-bf7c-cdda5ef25332 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.335394 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6105954-5d36-4f8e-bf7c-cdda5ef25332 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.335862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6105954-5d36-4f8e-bf7c-cdda5ef25332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.343626 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6105954-5d36-4f8e-bf7c-cdda5ef25332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:09.345611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6105954-5d36-4f8e-bf7c-cdda5ef25332 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:09.356545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7db2c-c2dc-4b88-8854-3b4173573142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:09.357470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7db2c-c2dc-4b88-8854-3b4173573142 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.357806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7db2c-c2dc-4b88-8854-3b4173573142 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.358089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7db2c-c2dc-4b88-8854-3b4173573142 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/a86de22b0e324c70aad792ec6b1a7e7d/roles/6b7e9f2fcda040b18213464589952d67` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.360604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b6105954-5d36-4f8e-bf7c-cdda5ef25332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:09.363836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:09.364678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:09.365630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:09.365630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:09.365630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:09.365630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:09.365630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:09.365630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:09.365630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:09.365630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:09.372599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 307/605] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:09.373881 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7db2c-c2dc-4b88-8854-3b4173573142 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=a86de22b0e324c70aad792ec6b1a7e7d, role_id=6b7e9f2fcda040b18213464589952d67)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:09.374593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad4b6e-bd7a-4c92-81e9-648c6c19b6c0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.375056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad4b6e-bd7a-4c92-81e9-648c6c19b6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.375504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad4b6e-bd7a-4c92-81e9-648c6c19b6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.375604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7db2c-c2dc-4b88-8854-3b4173573142 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:09.400761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 299/606] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:23:09 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/a86de22b0e324c70aad792ec6b1a7e7d/roles/6b7e9f2fcda040b18213464589952d67 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:09.400761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7db2c-c2dc-4b88-8854-3b4173573142 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:09.403289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-faad4b6e-bd7a-4c92-81e9-648c6c19b6c0 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:09.408683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-faad4b6e-bd7a-4c92-81e9-648c6c19b6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:09.538790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 308/607] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:09.570945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad4b6e-bd7a-4c92-81e9-648c6c19b6c0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:09.609620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7db2c-c2dc-4b88-8854-3b4173573142 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:09.647175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafef53-c4ee-4047-ba47-ac7731a0a244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:09.648220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafef53-c4ee-4047-ba47-ac7731a0a244 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.648714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafef53-c4ee-4047-ba47-ac7731a0a244 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.649234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafef53-c4ee-4047-ba47-ac7731a0a244 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.650323 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafef53-c4ee-4047-ba47-ac7731a0a244 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:09.650918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafef53-c4ee-4047-ba47-ac7731a0a244 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:09.652623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafef53-c4ee-4047-ba47-ac7731a0a244 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:09.669247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 300/608] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:09.673052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafef53-c4ee-4047-ba47-ac7731a0a244 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:09.683709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:09.687749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad4b6e-bd7a-4c92-81e9-648c6c19b6c0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:09.730722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc3ecf-735b-4da6-8754-1e6b0775530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:09.732400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc3ecf-735b-4da6-8754-1e6b0775530a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.732725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc3ecf-735b-4da6-8754-1e6b0775530a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.733048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc3ecf-735b-4da6-8754-1e6b0775530a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.744351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc3ecf-735b-4da6-8754-1e6b0775530a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-eacc3ecf-735b-4da6-8754-1e6b0775530a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:09.753055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:09.754058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:09.755267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 309/609] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:09.760426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc3ecf-735b-4da6-8754-1e6b0775530a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:09.761693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:09.850582 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafef53-c4ee-4047-ba47-ac7731a0a244 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:09.882054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc3ecf-735b-4da6-8754-1e6b0775530a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:09.887454 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd81154-6c34-4f16-9946-65f5d3f47247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:09.888461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd81154-6c34-4f16-9946-65f5d3f47247 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.889202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd81154-6c34-4f16-9946-65f5d3f47247 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.889325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd81154-6c34-4f16-9946-65f5d3f47247 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/5586ef8925ef4cc496d358283c21c962/roles/6b7e9f2fcda040b18213464589952d67` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.906467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd81154-6c34-4f16-9946-65f5d3f47247 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=5586ef8925ef4cc496d358283c21c962, role_id=6b7e9f2fcda040b18213464589952d67)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:09.908137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd81154-6c34-4f16-9946-65f5d3f47247 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:09.923156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c33f2-1ad5-41ee-935d-af21dc7faf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:09.923752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c33f2-1ad5-41ee-935d-af21dc7faf1e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.924036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c33f2-1ad5-41ee-935d-af21dc7faf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.924319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c33f2-1ad5-41ee-935d-af21dc7faf1e None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.925157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c33f2-1ad5-41ee-935d-af21dc7faf1e None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:09.925465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c33f2-1ad5-41ee-935d-af21dc7faf1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:09.926982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c33f2-1ad5-41ee-935d-af21dc7faf1e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:09.928804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 301/610] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:23:09 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/5586ef8925ef4cc496d358283c21c962/roles/6b7e9f2fcda040b18213464589952d67 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:23:09.930664 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f991457-596e-4545-ad1c-300d2385e669 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.930997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f991457-596e-4545-ad1c-300d2385e669 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.931273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f991457-596e-4545-ad1c-300d2385e669 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.933501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 302/611] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:09.936577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 310/612] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:09.938765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d8caf-1e86-4ab4-a1b3-e576a4fb72b4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:09.939254 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d8caf-1e86-4ab4-a1b3-e576a4fb72b4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:09.939657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d8caf-1e86-4ab4-a1b3-e576a4fb72b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:09.943879 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c33f2-1ad5-41ee-935d-af21dc7faf1e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:09.963324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-845d8caf-1e86-4ab4-a1b3-e576a4fb72b4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:09.964892 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-845d8caf-1e86-4ab4-a1b3-e576a4fb72b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:09.970239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:10.056811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c33f2-1ad5-41ee-935d-af21dc7faf1e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:10.065770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 303/613] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:23:10.094896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b24466-f2a3-454d-a9b8-5e8d33329b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:10.095770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b24466-f2a3-454d-a9b8-5e8d33329b14 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.096055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b24466-f2a3-454d-a9b8-5e8d33329b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.096324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b24466-f2a3-454d-a9b8-5e8d33329b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:10.102371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d8caf-1e86-4ab4-a1b3-e576a4fb72b4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:10.108700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b24466-f2a3-454d-a9b8-5e8d33329b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:10.110248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b24466-f2a3-454d-a9b8-5e8d33329b14 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-36b24466-f2a3-454d-a9b8-5e8d33329b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:10.128969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:10.131097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:10.133290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:10.133290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:10.133290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:10.133290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:10.133290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 311/614] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:10.140768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b24466-f2a3-454d-a9b8-5e8d33329b14 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:10.161115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:10.239873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:10.261564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d8caf-1e86-4ab4-a1b3-e576a4fb72b4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:10.276803 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b24466-f2a3-454d-a9b8-5e8d33329b14 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:10.298168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f043ecf-76d2-4b4c-84cf-bbe85acaa8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:10.299128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f043ecf-76d2-4b4c-84cf-bbe85acaa8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.299551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f043ecf-76d2-4b4c-84cf-bbe85acaa8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.299915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f043ecf-76d2-4b4c-84cf-bbe85acaa8a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:10.311341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f043ecf-76d2-4b4c-84cf-bbe85acaa8a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8f043ecf-76d2-4b4c-84cf-bbe85acaa8a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:10.319492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:10.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:10.321641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:10.323723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 304/615] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:10.325358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7219161-5d02-42f3-8634-efce29277c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:10.326037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7219161-5d02-42f3-8634-efce29277c6e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.326317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7219161-5d02-42f3-8634-efce29277c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.326639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7219161-5d02-42f3-8634-efce29277c6e None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:10.327465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7219161-5d02-42f3-8634-efce29277c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:10.327776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7219161-5d02-42f3-8634-efce29277c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:10.329219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7219161-5d02-42f3-8634-efce29277c6e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:10.329564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f043ecf-76d2-4b4c-84cf-bbe85acaa8a8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:10.350968 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 312/616] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:10.356272 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7219161-5d02-42f3-8634-efce29277c6e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:10.420374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:10.439012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f043ecf-76d2-4b4c-84cf-bbe85acaa8a8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:10.476270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61268c3-7985-4298-8102-3a98df5948d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:10.477340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61268c3-7985-4298-8102-3a98df5948d9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.477814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61268c3-7985-4298-8102-3a98df5948d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.478412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61268c3-7985-4298-8102-3a98df5948d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:10.479829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61268c3-7985-4298-8102-3a98df5948d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:10.480335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61268c3-7985-4298-8102-3a98df5948d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:10.482196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61268c3-7985-4298-8102-3a98df5948d9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:10.493902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 305/617] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:10.499254 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61268c3-7985-4298-8102-3a98df5948d9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:10.561071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7219161-5d02-42f3-8634-efce29277c6e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:10.639941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61268c3-7985-4298-8102-3a98df5948d9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:10.653811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:10.676594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b159e1-e33d-4fed-af84-6dfe84ba619f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:10.677963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b159e1-e33d-4fed-af84-6dfe84ba619f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.678440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b159e1-e33d-4fed-af84-6dfe84ba619f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.678903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b159e1-e33d-4fed-af84-6dfe84ba619f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:10.685460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d10225-9d52-474a-89bd-444954f859d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:10.686337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10225-9d52-474a-89bd-444954f859d5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.686609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10225-9d52-474a-89bd-444954f859d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.686888 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10225-9d52-474a-89bd-444954f859d5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:10.691372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b159e1-e33d-4fed-af84-6dfe84ba619f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:10.693192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b159e1-e33d-4fed-af84-6dfe84ba619f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:10.705933 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d10225-9d52-474a-89bd-444954f859d5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:10.707387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d10225-9d52-474a-89bd-444954f859d5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-51b159e1-e33d-4fed-af84-6dfe84ba619f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:10.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:10.712584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:10.713534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:10.713534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:10.713534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:10.713534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:10.715677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:10.716103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 306/618] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:10.723446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b159e1-e33d-4fed-af84-6dfe84ba619f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-85d10225-9d52-474a-89bd-444954f859d5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:10.733333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:10.734275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:10.735329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 313/619] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:10.740381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d10225-9d52-474a-89bd-444954f859d5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:10.844589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d10225-9d52-474a-89bd-444954f859d5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:10.889558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871a0d2-2e6e-41ad-97ae-796a4e958364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:10.890219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871a0d2-2e6e-41ad-97ae-796a4e958364 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.890504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871a0d2-2e6e-41ad-97ae-796a4e958364 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.890778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871a0d2-2e6e-41ad-97ae-796a4e958364 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:10.891624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871a0d2-2e6e-41ad-97ae-796a4e958364 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:10.891930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871a0d2-2e6e-41ad-97ae-796a4e958364 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:10.900756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871a0d2-2e6e-41ad-97ae-796a4e958364 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:10.917895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b159e1-e33d-4fed-af84-6dfe84ba619f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:10.918987 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 314/620] 10.222.0.69 () {56 vars in 1135 bytes} [Fri Jun 24 10:23:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:10.920472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ac913-7cf9-4c3c-97d4-9d1d6b58e484 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.920932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ac913-7cf9-4c3c-97d4-9d1d6b58e484 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.921118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ac913-7cf9-4c3c-97d4-9d1d6b58e484 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:10.922854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 315/621] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:10.925571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ac913-7cf9-4c3c-97d4-9d1d6b58e484 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:10.962668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3688dc2-f0fc-419f-882d-8c1fe3a36a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:10.963763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3688dc2-f0fc-419f-882d-8c1fe3a36a8f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.964221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3688dc2-f0fc-419f-882d-8c1fe3a36a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.964943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3688dc2-f0fc-419f-882d-8c1fe3a36a8f None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:10.966682 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3688dc2-f0fc-419f-882d-8c1fe3a36a8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:10.967212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3688dc2-f0fc-419f-882d-8c1fe3a36a8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:10.968774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3688dc2-f0fc-419f-882d-8c1fe3a36a8f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:10.983049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:10.985786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 307/622] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:10.990521 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7922-cedf-43bd-9f57-c2dca21c10f8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:10.990521 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7922-cedf-43bd-9f57-c2dca21c10f8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:10.991364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7922-cedf-43bd-9f57-c2dca21c10f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.011746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e1ad7922-cedf-43bd-9f57-c2dca21c10f8 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:11.012955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e1ad7922-cedf-43bd-9f57-c2dca21c10f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:11.054333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ac913-7cf9-4c3c-97d4-9d1d6b58e484 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:11.067398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:11.088820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 308/623] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:11.089744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7922-cedf-43bd-9f57-c2dca21c10f8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:11.094029 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e670-75c4-45d8-a254-c75a768f6d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:11.095163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e670-75c4-45d8-a254-c75a768f6d60 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:11.095644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e670-75c4-45d8-a254-c75a768f6d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:11.096130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e670-75c4-45d8-a254-c75a768f6d60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.107107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412e670-75c4-45d8-a254-c75a768f6d60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:11.108868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412e670-75c4-45d8-a254-c75a768f6d60 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3412e670-75c4-45d8-a254-c75a768f6d60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:11.127010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:11.127931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:11.128822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:11.128822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:11.128822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:11.128822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:11.131062 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 316/624] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:11.134785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e670-75c4-45d8-a254-c75a768f6d60 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:11.225360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7922-cedf-43bd-9f57-c2dca21c10f8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:11.252839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e670-75c4-45d8-a254-c75a768f6d60 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:11.254498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:11.272576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5bd1c-5327-4ebe-b305-7bfea383e173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:11.273720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5bd1c-5327-4ebe-b305-7bfea383e173 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:11.274196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5bd1c-5327-4ebe-b305-7bfea383e173 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:11.274693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5bd1c-5327-4ebe-b305-7bfea383e173 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.284526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5bd1c-5327-4ebe-b305-7bfea383e173 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:11.286186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5bd1c-5327-4ebe-b305-7bfea383e173 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:11.293292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d457a0-a481-40f6-b7cf-9795ca3f9386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:11.294367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d457a0-a481-40f6-b7cf-9795ca3f9386 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:11.294847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d457a0-a481-40f6-b7cf-9795ca3f9386 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:11.295358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d457a0-a481-40f6-b7cf-9795ca3f9386 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-45a5bd1c-5327-4ebe-b305-7bfea383e173 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:11.300996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:11.301941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:11.302853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 309/625] 10.222.0.69 () {56 vars in 1093 bytes} [Fri Jun 24 10:23:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:11.303405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5bd1c-5327-4ebe-b305-7bfea383e173 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:11.311002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d457a0-a481-40f6-b7cf-9795ca3f9386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-86d457a0-a481-40f6-b7cf-9795ca3f9386 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:11.321491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:11.322401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:11.323207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:11.331468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 317/626] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:11.331468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d457a0-a481-40f6-b7cf-9795ca3f9386 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:11.365392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:11.432709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5bd1c-5327-4ebe-b305-7bfea383e173 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:11.449097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d457a0-a481-40f6-b7cf-9795ca3f9386 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:11.478687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4235af-ee45-4a8d-9a0e-28a88bfeafb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:11.479908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4235af-ee45-4a8d-9a0e-28a88bfeafb9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:11.480335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4235af-ee45-4a8d-9a0e-28a88bfeafb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:11.481813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4235af-ee45-4a8d-9a0e-28a88bfeafb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.482551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4235af-ee45-4a8d-9a0e-28a88bfeafb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:11.483704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4235af-ee45-4a8d-9a0e-28a88bfeafb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:11.485192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4235af-ee45-4a8d-9a0e-28a88bfeafb9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:11.495191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef5361-f9d7-4bad-b526-9332f884936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:11.496402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef5361-f9d7-4bad-b526-9332f884936a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:11.496934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef5361-f9d7-4bad-b526-9332f884936a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:11.497408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef5361-f9d7-4bad-b526-9332f884936a None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.497837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 310/627] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:11.498672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef5361-f9d7-4bad-b526-9332f884936a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:11.499140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef5361-f9d7-4bad-b526-9332f884936a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:11.500821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef5361-f9d7-4bad-b526-9332f884936a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:11.503304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4235af-ee45-4a8d-9a0e-28a88bfeafb9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:11.525683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 318/628] 10.222.0.69 () {56 vars in 1133 bytes} [Fri Jun 24 10:23:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:11.526988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef5361-f9d7-4bad-b526-9332f884936a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:11.551921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:11.616382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4235af-ee45-4a8d-9a0e-28a88bfeafb9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:11.625952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:11.634608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef5361-f9d7-4bad-b526-9332f884936a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:11.653686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38043f-430a-49a7-be2e-6310139ffa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:11.654713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38043f-430a-49a7-be2e-6310139ffa70 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:11.655275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38043f-430a-49a7-be2e-6310139ffa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:11.655743 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38043f-430a-49a7-be2e-6310139ffa70 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.656706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38043f-430a-49a7-be2e-6310139ffa70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:11.657225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38043f-430a-49a7-be2e-6310139ffa70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:11.658775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38043f-430a-49a7-be2e-6310139ffa70 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:11.672768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 311/629] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:11.673971 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38043f-430a-49a7-be2e-6310139ffa70 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:11.674277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb658dff-aa97-4059-b3a7-56b97f77ad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:11.675071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb658dff-aa97-4059-b3a7-56b97f77ad26 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:11.675369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb658dff-aa97-4059-b3a7-56b97f77ad26 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:11.675609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb658dff-aa97-4059-b3a7-56b97f77ad26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.682918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb658dff-aa97-4059-b3a7-56b97f77ad26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:11.684401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb658dff-aa97-4059-b3a7-56b97f77ad26 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-fb658dff-aa97-4059-b3a7-56b97f77ad26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:11.698984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:11.699880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:11.700772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:11.700772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:11.700772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:11.700772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:11.700772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:11.700772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:11.700772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:11.700772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:11.704900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 319/630] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:11.706083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb658dff-aa97-4059-b3a7-56b97f77ad26 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:11.781488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38043f-430a-49a7-be2e-6310139ffa70 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:11.789093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:11.810555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb658dff-aa97-4059-b3a7-56b97f77ad26 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:11.821542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454090e-55c0-4cf2-b3d6-5b9a781a0614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:11.822674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454090e-55c0-4cf2-b3d6-5b9a781a0614 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:11.823137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454090e-55c0-4cf2-b3d6-5b9a781a0614 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:11.823712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454090e-55c0-4cf2-b3d6-5b9a781a0614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.845924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837f3dc-2d4a-4c6b-8541-d4f677ba6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:11.846721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837f3dc-2d4a-4c6b-8541-d4f677ba6de4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:11.847005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837f3dc-2d4a-4c6b-8541-d4f677ba6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:11.847281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837f3dc-2d4a-4c6b-8541-d4f677ba6de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:11.859286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837f3dc-2d4a-4c6b-8541-d4f677ba6de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:11.860723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837f3dc-2d4a-4c6b-8541-d4f677ba6de4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:11.867564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-9837f3dc-2d4a-4c6b-8541-d4f677ba6de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:11.880343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:11.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:11.882707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:11.882707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:11.882707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:11.882707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:11.882707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 320/631] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:11.882707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837f3dc-2d4a-4c6b-8541-d4f677ba6de4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:11.905865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f454090e-55c0-4cf2-b3d6-5b9a781a0614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:11.907370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f454090e-55c0-4cf2-b3d6-5b9a781a0614 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:11.993036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837f3dc-2d4a-4c6b-8541-d4f677ba6de4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.015322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 312/632] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:12.015322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4793-609c-4d05-89f9-d47119c4b00b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.015322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4793-609c-4d05-89f9-d47119c4b00b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.015322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4793-609c-4d05-89f9-d47119c4b00b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.015322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 313/633] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:12.015322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de4793-609c-4d05-89f9-d47119c4b00b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.031778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:12.038802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74d8a-30b3-433f-87cd-dd35593898e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.039883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74d8a-30b3-433f-87cd-dd35593898e8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.040170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74d8a-30b3-433f-87cd-dd35593898e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.040448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74d8a-30b3-433f-87cd-dd35593898e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.041417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b74d8a-30b3-433f-87cd-dd35593898e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:12.041727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b74d8a-30b3-433f-87cd-dd35593898e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:12.043156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b74d8a-30b3-433f-87cd-dd35593898e8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:12.055778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 321/634] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:12.056399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74d8a-30b3-433f-87cd-dd35593898e8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.098023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:12.131504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de4793-609c-4d05-89f9-d47119c4b00b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.161715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74d8a-30b3-433f-87cd-dd35593898e8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.169194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0d544-bd0f-4d27-a661-4f6325e213c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.170403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0d544-bd0f-4d27-a661-4f6325e213c7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.170850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0d544-bd0f-4d27-a661-4f6325e213c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.171304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0d544-bd0f-4d27-a661-4f6325e213c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.172421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0d544-bd0f-4d27-a661-4f6325e213c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:12.172760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0d544-bd0f-4d27-a661-4f6325e213c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:12.174496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0d544-bd0f-4d27-a661-4f6325e213c7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:12.187367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 314/635] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:12.188494 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9368e-884f-4fd8-9cfe-c96d24f6e510 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.188888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9368e-884f-4fd8-9cfe-c96d24f6e510 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.189130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9368e-884f-4fd8-9cfe-c96d24f6e510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.204171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb6c33-f6bb-4a4b-84b1-e6dd2d702306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.204988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb6c33-f6bb-4a4b-84b1-e6dd2d702306 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.205597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb6c33-f6bb-4a4b-84b1-e6dd2d702306 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.205866 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb6c33-f6bb-4a4b-84b1-e6dd2d702306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.206149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b1d9368e-884f-4fd8-9cfe-c96d24f6e510 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:12.207332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b1d9368e-884f-4fd8-9cfe-c96d24f6e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:12.216185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb6c33-f6bb-4a4b-84b1-e6dd2d702306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:12.217719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb6c33-f6bb-4a4b-84b1-e6dd2d702306 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0dbb6c33-f6bb-4a4b-84b1-e6dd2d702306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:12.236262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:12.237213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:12.238075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:12.238075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:12.238075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:12.238075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:12.238075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 322/636] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:12.238347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb6c33-f6bb-4a4b-84b1-e6dd2d702306 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.281173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 315/637] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:12.282021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9368e-884f-4fd8-9cfe-c96d24f6e510 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.299926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:12.356004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:12.357896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb6c33-f6bb-4a4b-84b1-e6dd2d702306 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.395157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ade7-9be1-4a1f-8a89-583f3dc6b435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.395972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ade7-9be1-4a1f-8a89-583f3dc6b435 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.396258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ade7-9be1-4a1f-8a89-583f3dc6b435 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.396537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ade7-9be1-4a1f-8a89-583f3dc6b435 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/44d5bd3f9dfe45ecb8d1a51d248f3c54/roles/6b7e9f2fcda040b18213464589952d67` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.397531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9368e-884f-4fd8-9cfe-c96d24f6e510 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.412936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ade7-9be1-4a1f-8a89-583f3dc6b435 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=44d5bd3f9dfe45ecb8d1a51d248f3c54, role_id=6b7e9f2fcda040b18213464589952d67)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:12.414397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ade7-9be1-4a1f-8a89-583f3dc6b435 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:12.427708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 323/638] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:23:12 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/44d5bd3f9dfe45ecb8d1a51d248f3c54/roles/6b7e9f2fcda040b18213464589952d67 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:12.428305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ade7-9be1-4a1f-8a89-583f3dc6b435 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.432956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a609cd-c7b9-438b-ac57-ad3801a39e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.433774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a609cd-c7b9-438b-ac57-ad3801a39e03 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.434057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a609cd-c7b9-438b-ac57-ad3801a39e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.434331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a609cd-c7b9-438b-ac57-ad3801a39e03 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.442836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a609cd-c7b9-438b-ac57-ad3801a39e03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:12.444319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a609cd-c7b9-438b-ac57-ad3801a39e03 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-91a609cd-c7b9-438b-ac57-ad3801a39e03 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:12.460450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:12.461446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:12.462356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 316/639] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:12.462932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a609cd-c7b9-438b-ac57-ad3801a39e03 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.553442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:12.561713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ade7-9be1-4a1f-8a89-583f3dc6b435 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.579455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a609cd-c7b9-438b-ac57-ad3801a39e03 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.604175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4e1c1-abbd-44b1-88fe-232f401dbcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.604872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4e1c1-abbd-44b1-88fe-232f401dbcbc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.605154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4e1c1-abbd-44b1-88fe-232f401dbcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.605421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:12.605558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4e1c1-abbd-44b1-88fe-232f401dbcbc None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.606147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4e1c1-abbd-44b1-88fe-232f401dbcbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:12.606505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4e1c1-abbd-44b1-88fe-232f401dbcbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:12.607882 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4e1c1-abbd-44b1-88fe-232f401dbcbc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:12.619719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 324/640] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:12.620831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23d5a1-ae1f-4d3f-b372-accf4638a5b7 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.621120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23d5a1-ae1f-4d3f-b372-accf4638a5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.621395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23d5a1-ae1f-4d3f-b372-accf4638a5b7 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.622791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 325/641] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:12.623337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23d5a1-ae1f-4d3f-b372-accf4638a5b7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.626594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8afc9-42ba-4434-9286-941613a5e8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.627686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8afc9-42ba-4434-9286-941613a5e8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.628192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8afc9-42ba-4434-9286-941613a5e8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.628639 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8afc9-42ba-4434-9286-941613a5e8e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.640079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8afc9-42ba-4434-9286-941613a5e8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-0cb8afc9-42ba-4434-9286-941613a5e8e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:12.648218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:12.649224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:12.650056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 317/642] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:12.650763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8afc9-42ba-4434-9286-941613a5e8e0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.728246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23d5a1-ae1f-4d3f-b372-accf4638a5b7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.758920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8afc9-42ba-4434-9286-941613a5e8e0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.767038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faa571-3996-48f1-b424-df50306117d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.767743 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faa571-3996-48f1-b424-df50306117d5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.768391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faa571-3996-48f1-b424-df50306117d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.768677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faa571-3996-48f1-b424-df50306117d5 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.769899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faa571-3996-48f1-b424-df50306117d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:12.770223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faa571-3996-48f1-b424-df50306117d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:12.771677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faa571-3996-48f1-b424-df50306117d5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:12.794302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 326/643] 10.222.0.69 () {56 vars in 1137 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:12.794302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a397a56-44fb-400a-879b-7c8aa26a2322 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.794302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a397a56-44fb-400a-879b-7c8aa26a2322 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.794302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a397a56-44fb-400a-879b-7c8aa26a2322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.799139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedc7e2-88a6-47e7-9efd-e82b427ff365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.799942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc7e2-88a6-47e7-9efd-e82b427ff365 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.800222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc7e2-88a6-47e7-9efd-e82b427ff365 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.800519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc7e2-88a6-47e7-9efd-e82b427ff365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.807911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedc7e2-88a6-47e7-9efd-e82b427ff365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:12.809005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3a397a56-44fb-400a-879b-7c8aa26a2322 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:12.809421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedc7e2-88a6-47e7-9efd-e82b427ff365 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:12.810176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3a397a56-44fb-400a-879b-7c8aa26a2322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:12.818366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8eedc7e2-88a6-47e7-9efd-e82b427ff365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:12.825848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:12.826775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 318/644] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:12.828027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedc7e2-88a6-47e7-9efd-e82b427ff365 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.870947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:12.896774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 327/645] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:12.896774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a397a56-44fb-400a-879b-7c8aa26a2322 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:12.940408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedc7e2-88a6-47e7-9efd-e82b427ff365 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:12.975333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7d94a-a7b0-41f5-ab22-8dca6dffeddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:12.976017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7d94a-a7b0-41f5-ab22-8dca6dffeddc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:12.976283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7d94a-a7b0-41f5-ab22-8dca6dffeddc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:12.976573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7d94a-a7b0-41f5-ab22-8dca6dffeddc None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:12.977420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7d94a-a7b0-41f5-ab22-8dca6dffeddc None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:12.977720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7d94a-a7b0-41f5-ab22-8dca6dffeddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:12.979108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7d94a-a7b0-41f5-ab22-8dca6dffeddc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:12.987624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 319/646] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:12.988428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7d94a-a7b0-41f5-ab22-8dca6dffeddc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.002919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a397a56-44fb-400a-879b-7c8aa26a2322 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.042213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af22ac-bf5c-49b1-9b7e-5f2a9ff749a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.042956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af22ac-bf5c-49b1-9b7e-5f2a9ff749a5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.043334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af22ac-bf5c-49b1-9b7e-5f2a9ff749a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.043686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af22ac-bf5c-49b1-9b7e-5f2a9ff749a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.057091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:13.087408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7d94a-a7b0-41f5-ab22-8dca6dffeddc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.088940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:13.119578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af22ac-bf5c-49b1-9b7e-5f2a9ff749a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:13.121267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af22ac-bf5c-49b1-9b7e-5f2a9ff749a5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:13.121835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388bacf-3441-4914-8b52-406a67972b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.122499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388bacf-3441-4914-8b52-406a67972b9d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.122779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388bacf-3441-4914-8b52-406a67972b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.123054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388bacf-3441-4914-8b52-406a67972b9d None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.123886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388bacf-3441-4914-8b52-406a67972b9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:13.124178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388bacf-3441-4914-8b52-406a67972b9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:13.125617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388bacf-3441-4914-8b52-406a67972b9d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:13.137673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 320/647] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:13.138468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388bacf-3441-4914-8b52-406a67972b9d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.226480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 328/648] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:13.227250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af22ac-bf5c-49b1-9b7e-5f2a9ff749a5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.255114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388bacf-3441-4914-8b52-406a67972b9d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.270873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:13.292492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1999e98-ceaf-4b84-850c-2ca9a32027d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.293531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1999e98-ceaf-4b84-850c-2ca9a32027d7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.293819 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1999e98-ceaf-4b84-850c-2ca9a32027d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.294111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1999e98-ceaf-4b84-850c-2ca9a32027d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.305765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1999e98-ceaf-4b84-850c-2ca9a32027d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d1999e98-ceaf-4b84-850c-2ca9a32027d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:13.313609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:13.314524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 321/649] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:13.315918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1999e98-ceaf-4b84-850c-2ca9a32027d7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.337065 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af22ac-bf5c-49b1-9b7e-5f2a9ff749a5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.342289 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:13.373077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502cb6e-2ba3-40b9-9e2e-af388adb0134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.373937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502cb6e-2ba3-40b9-9e2e-af388adb0134 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.374271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502cb6e-2ba3-40b9-9e2e-af388adb0134 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.374544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502cb6e-2ba3-40b9-9e2e-af388adb0134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.384686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502cb6e-2ba3-40b9-9e2e-af388adb0134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:13.386158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502cb6e-2ba3-40b9-9e2e-af388adb0134 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f502cb6e-2ba3-40b9-9e2e-af388adb0134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:13.404037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:13.405169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:13.406117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:13.406117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:13.406117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:13.406117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:13.406117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 329/650] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:13.406510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502cb6e-2ba3-40b9-9e2e-af388adb0134 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.427753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1999e98-ceaf-4b84-850c-2ca9a32027d7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.463084 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797cb93-26aa-4ff1-b860-120b085356ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.463836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797cb93-26aa-4ff1-b860-120b085356ae None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.464124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797cb93-26aa-4ff1-b860-120b085356ae None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.464399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797cb93-26aa-4ff1-b860-120b085356ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.465365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797cb93-26aa-4ff1-b860-120b085356ae None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=bec9fee282a64933aa0f6fbdd91a93c4) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:13.465574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797cb93-26aa-4ff1-b860-120b085356ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:13.467182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797cb93-26aa-4ff1-b860-120b085356ae None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:13.473316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 322/651] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/role_assignments?user.id=bec9fee282a64933aa0f6fbdd91a93c4&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:13.477588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797cb93-26aa-4ff1-b860-120b085356ae None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.519055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:13.527053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502cb6e-2ba3-40b9-9e2e-af388adb0134 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.568771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee9f19-8714-433f-9a1c-b387f4b8ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.569606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee9f19-8714-433f-9a1c-b387f4b8ab0c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.569891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee9f19-8714-433f-9a1c-b387f4b8ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.570170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee9f19-8714-433f-9a1c-b387f4b8ab0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.578832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:13.585156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee9f19-8714-433f-9a1c-b387f4b8ab0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:13.587092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee9f19-8714-433f-9a1c-b387f4b8ab0c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:13.596669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797cb93-26aa-4ff1-b860-120b085356ae None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a2ee9f19-8714-433f-9a1c-b387f4b8ab0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:13.606869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:13.608201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:13.609129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:13.609129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:13.609129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:13.609129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:13.609129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 330/652] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:13.609835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee9f19-8714-433f-9a1c-b387f4b8ab0c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.637471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f640dce-3f4f-4d33-ac36-52cd568421c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.638556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f640dce-3f4f-4d33-ac36-52cd568421c2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.639089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f640dce-3f4f-4d33-ac36-52cd568421c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.639576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f640dce-3f4f-4d33-ac36-52cd568421c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.640546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f640dce-3f4f-4d33-ac36-52cd568421c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:13.641119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f640dce-3f4f-4d33-ac36-52cd568421c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:13.642794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f640dce-3f4f-4d33-ac36-52cd568421c2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:13.652452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 323/653] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:13.657696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f640dce-3f4f-4d33-ac36-52cd568421c2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.723282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee9f19-8714-433f-9a1c-b387f4b8ab0c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.767376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1403d-51af-460f-bf43-277406d26197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.768197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1403d-51af-460f-bf43-277406d26197 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.768391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1403d-51af-460f-bf43-277406d26197 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.768721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1403d-51af-460f-bf43-277406d26197 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.769603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1403d-51af-460f-bf43-277406d26197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:13.769911 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1403d-51af-460f-bf43-277406d26197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:13.771277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1403d-51af-460f-bf43-277406d26197 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:13.781920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f640dce-3f4f-4d33-ac36-52cd568421c2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.784763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 331/654] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:13.786707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1403d-51af-460f-bf43-277406d26197 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.802107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:13.819393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2399881-bd36-40e8-9367-90f3fd88333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.820091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2399881-bd36-40e8-9367-90f3fd88333a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.820793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2399881-bd36-40e8-9367-90f3fd88333a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.821081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2399881-bd36-40e8-9367-90f3fd88333a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.822098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2399881-bd36-40e8-9367-90f3fd88333a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:13.822406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2399881-bd36-40e8-9367-90f3fd88333a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:13.824248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2399881-bd36-40e8-9367-90f3fd88333a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:13.837143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 324/655] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:13.838053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2399881-bd36-40e8-9367-90f3fd88333a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:13.853328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:13.915222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1403d-51af-460f-bf43-277406d26197 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.951808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ef555-32a1-49ad-a036-304cc4adc41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:13.952415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ef555-32a1-49ad-a036-304cc4adc41f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:13.952763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ef555-32a1-49ad-a036-304cc4adc41f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:13.953102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ef555-32a1-49ad-a036-304cc4adc41f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:13.965097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ef555-32a1-49ad-a036-304cc4adc41f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:13.966645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ef555-32a1-49ad-a036-304cc4adc41f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:13.971629 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2399881-bd36-40e8-9367-90f3fd88333a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ea5ef555-32a1-49ad-a036-304cc4adc41f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:13.991369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:13.992317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:13.993232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:13.993232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:13.993232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:13.993232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:13.993232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 332/656] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:13.998179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ef555-32a1-49ad-a036-304cc4adc41f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.014317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0bc8-ec2e-406b-a6e9-8069262009fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:14.015621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0bc8-ec2e-406b-a6e9-8069262009fe None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.016169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0bc8-ec2e-406b-a6e9-8069262009fe None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.016453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0bc8-ec2e-406b-a6e9-8069262009fe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.026397 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a0bc8-ec2e-406b-a6e9-8069262009fe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:14.028305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a0bc8-ec2e-406b-a6e9-8069262009fe None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c28a0bc8-ec2e-406b-a6e9-8069262009fe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:14.049085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:14.050981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:23:14.052239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:14.057085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 325/657] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:14.057085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0bc8-ec2e-406b-a6e9-8069262009fe None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.068687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:14.123401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ef555-32a1-49ad-a036-304cc4adc41f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:14.158217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:14.164889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fa2d1-6abb-4e82-92a4-aa48881b1657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:14.166515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fa2d1-6abb-4e82-92a4-aa48881b1657 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.167031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fa2d1-6abb-4e82-92a4-aa48881b1657 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.167589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fa2d1-6abb-4e82-92a4-aa48881b1657 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.175591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818fa2d1-6abb-4e82-92a4-aa48881b1657 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:14.177096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818fa2d1-6abb-4e82-92a4-aa48881b1657 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:14.188145 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0bc8-ec2e-406b-a6e9-8069262009fe None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-818fa2d1-6abb-4e82-92a4-aa48881b1657 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:14.193444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:14.194306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:14.195166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:14.195166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:14.195166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:14.195166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:14.195166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:14.195166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:14.195166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:14.195166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:14.199548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 333/658] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:14.200434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fa2d1-6abb-4e82-92a4-aa48881b1657 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.229583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb62af-c870-4178-8e5a-bf05b4e81b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:14.230735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb62af-c870-4178-8e5a-bf05b4e81b1a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.231216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb62af-c870-4178-8e5a-bf05b4e81b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.231687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb62af-c870-4178-8e5a-bf05b4e81b1a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.232760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb62af-c870-4178-8e5a-bf05b4e81b1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:14.233268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb62af-c870-4178-8e5a-bf05b4e81b1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:14.235013 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb62af-c870-4178-8e5a-bf05b4e81b1a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:14.247930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 326/659] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:14.253515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb62af-c870-4178-8e5a-bf05b4e81b1a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.308388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fa2d1-6abb-4e82-92a4-aa48881b1657 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:14.327933 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:14.348260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928d6d4-8b0b-49b2-93e1-4403faa6c56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:14.349417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928d6d4-8b0b-49b2-93e1-4403faa6c56b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.349417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928d6d4-8b0b-49b2-93e1-4403faa6c56b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.349607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928d6d4-8b0b-49b2-93e1-4403faa6c56b None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.350459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928d6d4-8b0b-49b2-93e1-4403faa6c56b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:14.350825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928d6d4-8b0b-49b2-93e1-4403faa6c56b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:14.352339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928d6d4-8b0b-49b2-93e1-4403faa6c56b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:14.363060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb62af-c870-4178-8e5a-bf05b4e81b1a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:14.372318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 334/660] 10.222.0.69 () {56 vars in 1139 bytes} [Fri Jun 24 10:23:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:14.373486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928d6d4-8b0b-49b2-93e1-4403faa6c56b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.391999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:14.400707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9f329-e0bb-4a05-aad1-b542b2a6dd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:14.401809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9f329-e0bb-4a05-aad1-b542b2a6dd33 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.402296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9f329-e0bb-4a05-aad1-b542b2a6dd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.402832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9f329-e0bb-4a05-aad1-b542b2a6dd33 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.403838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9f329-e0bb-4a05-aad1-b542b2a6dd33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:14.404356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9f329-e0bb-4a05-aad1-b542b2a6dd33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:14.406231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9f329-e0bb-4a05-aad1-b542b2a6dd33 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:14.419703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 327/661] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:14.421021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9f329-e0bb-4a05-aad1-b542b2a6dd33 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.494958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928d6d4-8b0b-49b2-93e1-4403faa6c56b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:14.531154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc27d1c-173d-4ac6-8645-7d00c8de67b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:14.531960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc27d1c-173d-4ac6-8645-7d00c8de67b2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.532247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc27d1c-173d-4ac6-8645-7d00c8de67b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.532528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc27d1c-173d-4ac6-8645-7d00c8de67b2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.533774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9f329-e0bb-4a05-aad1-b542b2a6dd33 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:14.539691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc27d1c-173d-4ac6-8645-7d00c8de67b2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:14.541162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc27d1c-173d-4ac6-8645-7d00c8de67b2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-9cc27d1c-173d-4ac6-8645-7d00c8de67b2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:14.555409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:14.556421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 335/662] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:14.557682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc27d1c-173d-4ac6-8645-7d00c8de67b2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.574567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149769ec-26d6-4991-a10d-a6122bafc62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:14.575156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149769ec-26d6-4991-a10d-a6122bafc62e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149769ec-26d6-4991-a10d-a6122bafc62e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.575953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149769ec-26d6-4991-a10d-a6122bafc62e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.595962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:14.635014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:14.649149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149769ec-26d6-4991-a10d-a6122bafc62e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:14.650616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149769ec-26d6-4991-a10d-a6122bafc62e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:14.663936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc27d1c-173d-4ac6-8645-7d00c8de67b2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:14.701360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cac782-2de3-4532-ba02-a54c25bd9b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:14.702105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cac782-2de3-4532-ba02-a54c25bd9b3c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.702387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cac782-2de3-4532-ba02-a54c25bd9b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.702661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cac782-2de3-4532-ba02-a54c25bd9b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.703497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cac782-2de3-4532-ba02-a54c25bd9b3c None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=1b75a27126c9464989cec70e5104824c) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:14.703811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cac782-2de3-4532-ba02-a54c25bd9b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:14.705605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cac782-2de3-4532-ba02-a54c25bd9b3c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:14.711784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 336/663] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:14 2022] GET /identity/v3/role_assignments?user.id=1b75a27126c9464989cec70e5104824c&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:14.713032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b78f3-600d-4247-8802-b697f14dfe83 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.713436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b78f3-600d-4247-8802-b697f14dfe83 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.713716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b78f3-600d-4247-8802-b697f14dfe83 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.716807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 337/664] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:14.716807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b78f3-600d-4247-8802-b697f14dfe83 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.791349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 328/665] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:14.793663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523d233-a48a-46d0-a0f3-cc4cf816de43 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.794128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523d233-a48a-46d0-a0f3-cc4cf816de43 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.795026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523d233-a48a-46d0-a0f3-cc4cf816de43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.820527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3523d233-a48a-46d0-a0f3-cc4cf816de43 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:14.822292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3523d233-a48a-46d0-a0f3-cc4cf816de43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:14.841846 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:14.847082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b78f3-600d-4247-8802-b697f14dfe83 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:14.896607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff255f-3e27-4b55-9dc5-63ed4baf827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:14.897742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff255f-3e27-4b55-9dc5-63ed4baf827d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:14.898023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff255f-3e27-4b55-9dc5-63ed4baf827d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:14.898443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff255f-3e27-4b55-9dc5-63ed4baf827d None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:14.899399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff255f-3e27-4b55-9dc5-63ed4baf827d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:14.899859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff255f-3e27-4b55-9dc5-63ed4baf827d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:14.902016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff255f-3e27-4b55-9dc5-63ed4baf827d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:14.911568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 329/666] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:14.912718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523d233-a48a-46d0-a0f3-cc4cf816de43 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.926916 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 338/667] 10.222.0.69 () {56 vars in 1137 bytes} [Fri Jun 24 10:23:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:14.933604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff255f-3e27-4b55-9dc5-63ed4baf827d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:14.938162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:15.034970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523d233-a48a-46d0-a0f3-cc4cf816de43 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:15.054325 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff255f-3e27-4b55-9dc5-63ed4baf827d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:15.077002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270db9b0-58d8-473a-96c4-de7d47e8938e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:15.077994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270db9b0-58d8-473a-96c4-de7d47e8938e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.078426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270db9b0-58d8-473a-96c4-de7d47e8938e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.078720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270db9b0-58d8-473a-96c4-de7d47e8938e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.092211 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20effb07-3694-45e7-a5a3-93f661752c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:15.093585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20effb07-3694-45e7-a5a3-93f661752c84 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.093878 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20effb07-3694-45e7-a5a3-93f661752c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.094160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20effb07-3694-45e7-a5a3-93f661752c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.094454 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270db9b0-58d8-473a-96c4-de7d47e8938e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:15.096205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270db9b0-58d8-473a-96c4-de7d47e8938e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-270db9b0-58d8-473a-96c4-de7d47e8938e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:15.116050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:15.117016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:15.117878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:15.117878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:15.117878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:15.117878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:15.120323 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 330/668] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:15.121430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270db9b0-58d8-473a-96c4-de7d47e8938e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:15.124391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:15.176992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20effb07-3694-45e7-a5a3-93f661752c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:15.178821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20effb07-3694-45e7-a5a3-93f661752c84 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:15.211552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:15.253701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270db9b0-58d8-473a-96c4-de7d47e8938e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:15.291924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 339/669] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:15.293169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20effb07-3694-45e7-a5a3-93f661752c84 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:15.301464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb95b3-0151-4a4c-a0b5-0b4354deba26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:15.302279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb95b3-0151-4a4c-a0b5-0b4354deba26 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.302569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb95b3-0151-4a4c-a0b5-0b4354deba26 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.302847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb95b3-0151-4a4c-a0b5-0b4354deba26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.316838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb95b3-0151-4a4c-a0b5-0b4354deba26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-29fb95b3-0151-4a4c-a0b5-0b4354deba26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:15.327106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:15.328064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:15.328938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:15.329562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 331/670] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:15.330436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb95b3-0151-4a4c-a0b5-0b4354deba26 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:15.389319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:15.406174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20effb07-3694-45e7-a5a3-93f661752c84 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:15.452370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb95b3-0151-4a4c-a0b5-0b4354deba26 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:15.458263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad680784-b845-4d11-af52-4f35a11b9660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:15.460050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad680784-b845-4d11-af52-4f35a11b9660 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.460524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad680784-b845-4d11-af52-4f35a11b9660 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.461016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad680784-b845-4d11-af52-4f35a11b9660 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.461948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad680784-b845-4d11-af52-4f35a11b9660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:15.462441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad680784-b845-4d11-af52-4f35a11b9660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:15.464508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad680784-b845-4d11-af52-4f35a11b9660 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:15.484633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 340/671] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:15.484633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad680784-b845-4d11-af52-4f35a11b9660 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:15.494193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:15.503095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c1850-3daa-4e45-bf5a-3ba4c461f741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:15.504689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c1850-3daa-4e45-bf5a-3ba4c461f741 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.505188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c1850-3daa-4e45-bf5a-3ba4c461f741 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.505987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c1850-3daa-4e45-bf5a-3ba4c461f741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.522430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c1850-3daa-4e45-bf5a-3ba4c461f741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:15.524905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c1850-3daa-4e45-bf5a-3ba4c461f741 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-dc7c1850-3daa-4e45-bf5a-3ba4c461f741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:15.547708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:15.548939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:15.550197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:15.550197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:15.550197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:15.550197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:15.552873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 332/672] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:15.553899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c1850-3daa-4e45-bf5a-3ba4c461f741 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:15.603668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad680784-b845-4d11-af52-4f35a11b9660 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:15.625980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:15.650193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa48e52-7ef0-4795-92c6-a02f6f325d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:15.651250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa48e52-7ef0-4795-92c6-a02f6f325d54 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.651751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa48e52-7ef0-4795-92c6-a02f6f325d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.652229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa48e52-7ef0-4795-92c6-a02f6f325d54 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.653621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa48e52-7ef0-4795-92c6-a02f6f325d54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:15.654169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa48e52-7ef0-4795-92c6-a02f6f325d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:15.656160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa48e52-7ef0-4795-92c6-a02f6f325d54 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:15.667187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 341/673] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:15.668640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa48e52-7ef0-4795-92c6-a02f6f325d54 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:15.682888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c1850-3daa-4e45-bf5a-3ba4c461f741 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:15.731197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40240cdf-fa1e-47a7-a7da-2221d129467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:15.732007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40240cdf-fa1e-47a7-a7da-2221d129467f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.732294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40240cdf-fa1e-47a7-a7da-2221d129467f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.732590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40240cdf-fa1e-47a7-a7da-2221d129467f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.742746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40240cdf-fa1e-47a7-a7da-2221d129467f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:15.744457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40240cdf-fa1e-47a7-a7da-2221d129467f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-40240cdf-fa1e-47a7-a7da-2221d129467f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:15.762894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:15.763841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:15.765393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:15.765393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:15.765393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:15.765393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:15.765393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:15.765393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:15.765393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:15.765393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:15.766134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 333/674] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:15.767017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40240cdf-fa1e-47a7-a7da-2221d129467f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:15.786132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa48e52-7ef0-4795-92c6-a02f6f325d54 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:15.811881 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:15.824218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8e7ba-1c9a-412b-aa00-e15f59df1e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:15.825162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8e7ba-1c9a-412b-aa00-e15f59df1e25 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.825453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8e7ba-1c9a-412b-aa00-e15f59df1e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.825719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8e7ba-1c9a-412b-aa00-e15f59df1e25 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.826600 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8e7ba-1c9a-412b-aa00-e15f59df1e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:15.826963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8e7ba-1c9a-412b-aa00-e15f59df1e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:15.828333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8e7ba-1c9a-412b-aa00-e15f59df1e25 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:15.840823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 342/675] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:15.842620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f1371-eff2-4e22-aec6-2255bdba8fcb None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.842620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f1371-eff2-4e22-aec6-2255bdba8fcb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.842835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f1371-eff2-4e22-aec6-2255bdba8fcb None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.844111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 343/676] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:15.845843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f1371-eff2-4e22-aec6-2255bdba8fcb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:15.880990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40240cdf-fa1e-47a7-a7da-2221d129467f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:15.906078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:15.917632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a904401-6834-4b33-8656-b10e7b08109c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:15.918448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a904401-6834-4b33-8656-b10e7b08109c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:15.918730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a904401-6834-4b33-8656-b10e7b08109c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:15.918970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a904401-6834-4b33-8656-b10e7b08109c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:15.930307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a904401-6834-4b33-8656-b10e7b08109c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:15.932103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a904401-6834-4b33-8656-b10e7b08109c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3a904401-6834-4b33-8656-b10e7b08109c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:15.959683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:15.960708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:15.961637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:15.961637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:15.961637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:15.961637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:15.961637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 334/677] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:15.961924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a904401-6834-4b33-8656-b10e7b08109c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:15.976071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f1371-eff2-4e22-aec6-2255bdba8fcb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.014660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20293983-a168-422e-a961-3ca530c33588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.015328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20293983-a168-422e-a961-3ca530c33588 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.015618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20293983-a168-422e-a961-3ca530c33588 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.015891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20293983-a168-422e-a961-3ca530c33588 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.016684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20293983-a168-422e-a961-3ca530c33588 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:16.017028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20293983-a168-422e-a961-3ca530c33588 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.018491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20293983-a168-422e-a961-3ca530c33588 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.030654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 344/678] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:16.031886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4bb06-49cf-4a66-a46c-c57db956c5e1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.032168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4bb06-49cf-4a66-a46c-c57db956c5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.032444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4bb06-49cf-4a66-a46c-c57db956c5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.049676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4ad4bb06-49cf-4a66-a46c-c57db956c5e1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:16.050974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4ad4bb06-49cf-4a66-a46c-c57db956c5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:16.073900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:16.082703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a904401-6834-4b33-8656-b10e7b08109c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.119366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817472e-d855-4380-b0e8-50af6f83ece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.119817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 345/679] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:16.120657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4bb06-49cf-4a66-a46c-c57db956c5e1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.121402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817472e-d855-4380-b0e8-50af6f83ece0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.121898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817472e-d855-4380-b0e8-50af6f83ece0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.122393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817472e-d855-4380-b0e8-50af6f83ece0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.136321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817472e-d855-4380-b0e8-50af6f83ece0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.138270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817472e-d855-4380-b0e8-50af6f83ece0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.152494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7817472e-d855-4380-b0e8-50af6f83ece0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:16.153054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:16.153922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:16.154830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:16.154830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:16.154830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:16.154830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:16.154830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:16.154830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:16.154830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:16.154830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:16.154830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 335/680] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:16.155316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817472e-d855-4380-b0e8-50af6f83ece0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.238196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4bb06-49cf-4a66-a46c-c57db956c5e1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.258108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817472e-d855-4380-b0e8-50af6f83ece0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.274126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced5245-1df8-4efe-9b89-6f016ccef84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.274840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced5245-1df8-4efe-9b89-6f016ccef84d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.275137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced5245-1df8-4efe-9b89-6f016ccef84d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.275400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced5245-1df8-4efe-9b89-6f016ccef84d None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.276127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced5245-1df8-4efe-9b89-6f016ccef84d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:16.276433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced5245-1df8-4efe-9b89-6f016ccef84d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.277838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced5245-1df8-4efe-9b89-6f016ccef84d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.289226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 346/681] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:16.290018 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced5245-1df8-4efe-9b89-6f016ccef84d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.297071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380432f-a0f2-450e-bb7c-5e14dd6b63c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.297735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380432f-a0f2-450e-bb7c-5e14dd6b63c6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.298016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380432f-a0f2-450e-bb7c-5e14dd6b63c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.298290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380432f-a0f2-450e-bb7c-5e14dd6b63c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.299124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380432f-a0f2-450e-bb7c-5e14dd6b63c6 None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=a86de22b0e324c70aad792ec6b1a7e7d) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:16.299435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380432f-a0f2-450e-bb7c-5e14dd6b63c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.300901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380432f-a0f2-450e-bb7c-5e14dd6b63c6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.316780 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 336/682] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/role_assignments?user.id=a86de22b0e324c70aad792ec6b1a7e7d&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:16.316780 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380432f-a0f2-450e-bb7c-5e14dd6b63c6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.340473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:16.404149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:16.415736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced5245-1df8-4efe-9b89-6f016ccef84d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.460114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380432f-a0f2-450e-bb7c-5e14dd6b63c6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.463796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35d4ad-3e64-4fd5-acc2-ca067eef0151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.464877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35d4ad-3e64-4fd5-acc2-ca067eef0151 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.465308 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35d4ad-3e64-4fd5-acc2-ca067eef0151 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.465724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35d4ad-3e64-4fd5-acc2-ca067eef0151 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.478787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35d4ad-3e64-4fd5-acc2-ca067eef0151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ca35d4ad-3e64-4fd5-acc2-ca067eef0151 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:16.488628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:16.489793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:16.490744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:16.492797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 347/683] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:16.493744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35d4ad-3e64-4fd5-acc2-ca067eef0151 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.501172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c1c7-afcd-47d8-be01-1387ed920a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.502084 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c1c7-afcd-47d8-be01-1387ed920a80 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.502530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c1c7-afcd-47d8-be01-1387ed920a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.502810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c1c7-afcd-47d8-be01-1387ed920a80 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.503649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33c1c7-afcd-47d8-be01-1387ed920a80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:16.503961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33c1c7-afcd-47d8-be01-1387ed920a80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.505868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33c1c7-afcd-47d8-be01-1387ed920a80 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.523928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 337/684] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:16.524710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c1c7-afcd-47d8-be01-1387ed920a80 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.618984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:16.628693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35d4ad-3e64-4fd5-acc2-ca067eef0151 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.649230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c1c7-afcd-47d8-be01-1387ed920a80 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.666308 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb1a95-978d-4a38-b863-420bdb294080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.667219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb1a95-978d-4a38-b863-420bdb294080 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.667710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb1a95-978d-4a38-b863-420bdb294080 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.667989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb1a95-978d-4a38-b863-420bdb294080 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.680060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb1a95-978d-4a38-b863-420bdb294080 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.682413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb1a95-978d-4a38-b863-420bdb294080 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.686730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470b72e-41aa-4958-b08a-8dd9903c9f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.687576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470b72e-41aa-4958-b08a-8dd9903c9f9b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.687952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470b72e-41aa-4958-b08a-8dd9903c9f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.688353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470b72e-41aa-4958-b08a-8dd9903c9f9b None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.689271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470b72e-41aa-4958-b08a-8dd9903c9f9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:16.689650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470b72e-41aa-4958-b08a-8dd9903c9f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.691104 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470b72e-41aa-4958-b08a-8dd9903c9f9b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-40eb1a95-978d-4a38-b863-420bdb294080 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:16.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:16.700641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:16.701852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 348/685] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:16.702572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb1a95-978d-4a38-b863-420bdb294080 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.707782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 338/686] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:16.708801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470b72e-41aa-4958-b08a-8dd9903c9f9b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.721115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:16.814985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470b72e-41aa-4958-b08a-8dd9903c9f9b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.835271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb1a95-978d-4a38-b863-420bdb294080 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.854421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119af153-87c2-4140-9081-f3903dd7036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.855432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119af153-87c2-4140-9081-f3903dd7036d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.855903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119af153-87c2-4140-9081-f3903dd7036d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.856410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119af153-87c2-4140-9081-f3903dd7036d None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.857540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119af153-87c2-4140-9081-f3903dd7036d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:16.858094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119af153-87c2-4140-9081-f3903dd7036d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.859916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119af153-87c2-4140-9081-f3903dd7036d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.871132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:16.878343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870d59f-59b9-4ab9-bea3-f145870fb37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.879289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870d59f-59b9-4ab9-bea3-f145870fb37b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.879576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870d59f-59b9-4ab9-bea3-f145870fb37b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.879855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870d59f-59b9-4ab9-bea3-f145870fb37b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.880994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870d59f-59b9-4ab9-bea3-f145870fb37b None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=5586ef8925ef4cc496d358283c21c962) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:16.881315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870d59f-59b9-4ab9-bea3-f145870fb37b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.882761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870d59f-59b9-4ab9-bea3-f145870fb37b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.883724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 339/687] 10.222.0.69 () {56 vars in 1135 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:16.884854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119af153-87c2-4140-9081-f3903dd7036d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.896775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 349/688] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/role_assignments?user.id=5586ef8925ef4cc496d358283c21c962&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:16.896775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870d59f-59b9-4ab9-bea3-f145870fb37b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.986806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:17.016620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870d59f-59b9-4ab9-bea3-f145870fb37b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:17.043854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119af153-87c2-4140-9081-f3903dd7036d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:17.062865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf721d03-e8f4-44df-b9c0-425ecd8041f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:17.064087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721d03-e8f4-44df-b9c0-425ecd8041f6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.064567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721d03-e8f4-44df-b9c0-425ecd8041f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.065335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721d03-e8f4-44df-b9c0-425ecd8041f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.093863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b955c7d-cb47-40c0-9d76-ba7f042b3057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:17.094684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b955c7d-cb47-40c0-9d76-ba7f042b3057 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.094968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b955c7d-cb47-40c0-9d76-ba7f042b3057 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.095490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b955c7d-cb47-40c0-9d76-ba7f042b3057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.107157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b955c7d-cb47-40c0-9d76-ba7f042b3057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:17.109485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b955c7d-cb47-40c0-9d76-ba7f042b3057 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3b955c7d-cb47-40c0-9d76-ba7f042b3057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:17.130685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:17.131648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:17.132523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:17.132523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:17.132523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:17.132523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:17.132523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 340/689] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:17.133558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461f92f-0c85-4e62-9b6e-800b6cd1e87f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.133932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461f92f-0c85-4e62-9b6e-800b6cd1e87f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.134213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461f92f-0c85-4e62-9b6e-800b6cd1e87f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.135796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 341/690] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:17.138443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461f92f-0c85-4e62-9b6e-800b6cd1e87f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:17.146599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf721d03-e8f4-44df-b9c0-425ecd8041f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:17.148241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf721d03-e8f4-44df-b9c0-425ecd8041f6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:17.204862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:17.236471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:17.266250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461f92f-0c85-4e62-9b6e-800b6cd1e87f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:17.274659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 350/691] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:17.276154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c410eaa-6c70-40cf-8af8-9b68e19ab082 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.276544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c410eaa-6c70-40cf-8af8-9b68e19ab082 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.276951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c410eaa-6c70-40cf-8af8-9b68e19ab082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.297992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4c410eaa-6c70-40cf-8af8-9b68e19ab082 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:17.299173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4c410eaa-6c70-40cf-8af8-9b68e19ab082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:17.316839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8369b8-4c50-485c-9438-1a1853693e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:17.317784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8369b8-4c50-485c-9438-1a1853693e5c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.318524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8369b8-4c50-485c-9438-1a1853693e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.318902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8369b8-4c50-485c-9438-1a1853693e5c None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.319738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8369b8-4c50-485c-9438-1a1853693e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:17.320146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8369b8-4c50-485c-9438-1a1853693e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:17.322140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8369b8-4c50-485c-9438-1a1853693e5c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:17.339860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 342/692] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:17.340717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8369b8-4c50-485c-9438-1a1853693e5c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:17.375784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 351/693] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:17.376850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c410eaa-6c70-40cf-8af8-9b68e19ab082 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:17.479453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8369b8-4c50-485c-9438-1a1853693e5c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:17.493740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:17.517661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c410eaa-6c70-40cf-8af8-9b68e19ab082 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:17.520728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:17.529672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a256a-95b8-4ae5-9f0b-4031eb45b955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:17.531465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a256a-95b8-4ae5-9f0b-4031eb45b955 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.531902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a256a-95b8-4ae5-9f0b-4031eb45b955 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.532326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a256a-95b8-4ae5-9f0b-4031eb45b955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.547794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a256a-95b8-4ae5-9f0b-4031eb45b955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:17.550054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a256a-95b8-4ae5-9f0b-4031eb45b955 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:17.555831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b33ae-aac8-4285-b1d1-a38aa14f6b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:17.556775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b33ae-aac8-4285-b1d1-a38aa14f6b09 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.557069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b33ae-aac8-4285-b1d1-a38aa14f6b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.557347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b33ae-aac8-4285-b1d1-a38aa14f6b09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.567775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b33ae-aac8-4285-b1d1-a38aa14f6b09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:17.569253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b33ae-aac8-4285-b1d1-a38aa14f6b09 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5b5a256a-95b8-4ae5-9f0b-4031eb45b955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:17.575440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:17.576510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:17.577513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:17.577513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:17.577513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:17.577513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:17.579681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 343/694] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:17.581078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a256a-95b8-4ae5-9f0b-4031eb45b955 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a14b33ae-aac8-4285-b1d1-a38aa14f6b09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:17.587657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:17.588757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:17.589862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 352/695] 10.222.0.69 () {56 vars in 1093 bytes} [Fri Jun 24 10:23:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:17.590449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b33ae-aac8-4285-b1d1-a38aa14f6b09 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:17.713520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a256a-95b8-4ae5-9f0b-4031eb45b955 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:17.718729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b33ae-aac8-4285-b1d1-a38aa14f6b09 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:17.748311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93523cc-9621-4e4b-a3ec-91ad676bb1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:17.749365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93523cc-9621-4e4b-a3ec-91ad676bb1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.749782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93523cc-9621-4e4b-a3ec-91ad676bb1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.750217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93523cc-9621-4e4b-a3ec-91ad676bb1f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.761450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93523cc-9621-4e4b-a3ec-91ad676bb1f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:17.761706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330be001-706e-4d9d-a5aa-16fae64b16d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:17.762371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330be001-706e-4d9d-a5aa-16fae64b16d9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.762654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330be001-706e-4d9d-a5aa-16fae64b16d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.762931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330be001-706e-4d9d-a5aa-16fae64b16d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.763653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330be001-706e-4d9d-a5aa-16fae64b16d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:17.763994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93523cc-9621-4e4b-a3ec-91ad676bb1f2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:17.764100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330be001-706e-4d9d-a5aa-16fae64b16d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:17.765504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330be001-706e-4d9d-a5aa-16fae64b16d9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:17.780197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 353/696] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:17.781056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330be001-706e-4d9d-a5aa-16fae64b16d9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b93523cc-9621-4e4b-a3ec-91ad676bb1f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:17.784497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:17.785594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:17.786616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:17.786616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:17.786616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:17.786616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:17.792786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 344/697] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:17.793762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93523cc-9621-4e4b-a3ec-91ad676bb1f2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:17.809516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:17.811206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:17.903348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330be001-706e-4d9d-a5aa-16fae64b16d9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:17.921707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93523cc-9621-4e4b-a3ec-91ad676bb1f2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:17.942863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e74c36-c217-4031-8fc1-1cc4af2fbcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:17.943551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e74c36-c217-4031-8fc1-1cc4af2fbcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.943839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e74c36-c217-4031-8fc1-1cc4af2fbcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.944117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e74c36-c217-4031-8fc1-1cc4af2fbcb3 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.945510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e74c36-c217-4031-8fc1-1cc4af2fbcb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:17.945829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e74c36-c217-4031-8fc1-1cc4af2fbcb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:17.947545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e74c36-c217-4031-8fc1-1cc4af2fbcb3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:17.960366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a7ef3-d3b7-43f8-a14e-1ea31a580a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:17.961466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a7ef3-d3b7-43f8-a14e-1ea31a580a48 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:17.961952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a7ef3-d3b7-43f8-a14e-1ea31a580a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:17.962404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a7ef3-d3b7-43f8-a14e-1ea31a580a48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:17.974165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a7ef3-d3b7-43f8-a14e-1ea31a580a48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:17.975906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a7ef3-d3b7-43f8-a14e-1ea31a580a48 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:17.977392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 354/698] 10.222.0.69 () {56 vars in 1133 bytes} [Fri Jun 24 10:23:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:17.978537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e74c36-c217-4031-8fc1-1cc4af2fbcb3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7e7a7ef3-d3b7-43f8-a14e-1ea31a580a48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:17.999347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:18.000279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:18.001180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:18.001180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:18.001180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:18.001180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:18.001180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:18.001180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:18.001180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:18.001180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:18.004137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 345/699] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:18.005252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a7ef3-d3b7-43f8-a14e-1ea31a580a48 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:18.074056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:18.093378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e74c36-c217-4031-8fc1-1cc4af2fbcb3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:18.093680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:18.121070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a7ef3-d3b7-43f8-a14e-1ea31a580a48 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:18.129129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5832c0-56c2-4781-8b67-3079522bc2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:18.129809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5832c0-56c2-4781-8b67-3079522bc2ef None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.130144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5832c0-56c2-4781-8b67-3079522bc2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.130428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5832c0-56c2-4781-8b67-3079522bc2ef None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.131173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5832c0-56c2-4781-8b67-3079522bc2ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:18.131484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5832c0-56c2-4781-8b67-3079522bc2ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:18.132927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5832c0-56c2-4781-8b67-3079522bc2ef None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:18.146181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 355/700] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:18.147124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5832c0-56c2-4781-8b67-3079522bc2ef None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:18.162433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f0b7c-b309-4759-bd44-b2a0de43c88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:18.163098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f0b7c-b309-4759-bd44-b2a0de43c88a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.163384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f0b7c-b309-4759-bd44-b2a0de43c88a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.163665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f0b7c-b309-4759-bd44-b2a0de43c88a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.238421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f0b7c-b309-4759-bd44-b2a0de43c88a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:18.239896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f0b7c-b309-4759-bd44-b2a0de43c88a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:18.248789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5832c0-56c2-4781-8b67-3079522bc2ef None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:18.286274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c4868-58c0-4856-983e-92728d46822c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:18.286942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c4868-58c0-4856-983e-92728d46822c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.287258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c4868-58c0-4856-983e-92728d46822c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.287604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c4868-58c0-4856-983e-92728d46822c None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.288404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c4868-58c0-4856-983e-92728d46822c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:18.288724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c4868-58c0-4856-983e-92728d46822c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:18.290292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c4868-58c0-4856-983e-92728d46822c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:18.308278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 356/701] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:18.309154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c4868-58c0-4856-983e-92728d46822c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:18.321380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:18.327583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:18.354110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 346/702] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:18.354110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f0b7c-b309-4759-bd44-b2a0de43c88a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:18.423518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c4868-58c0-4856-983e-92728d46822c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:18.475431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a1c1-c6e6-4a04-a542-d0451b11ea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:18.475858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f0b7c-b309-4759-bd44-b2a0de43c88a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:18.476122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a1c1-c6e6-4a04-a542-d0451b11ea03 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.476410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a1c1-c6e6-4a04-a542-d0451b11ea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.476692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a1c1-c6e6-4a04-a542-d0451b11ea03 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.477435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a1c1-c6e6-4a04-a542-d0451b11ea03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:18.480022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a1c1-c6e6-4a04-a542-d0451b11ea03 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:18.505197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 357/703] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:23:18 2022] POST /identity/v3/users => generated 254 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:18.506348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a1c1-c6e6-4a04-a542-d0451b11ea03 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:18.521853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb27625-5e77-46db-8d0d-11b31089d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:18.522529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb27625-5e77-46db-8d0d-11b31089d24a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.522869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb27625-5e77-46db-8d0d-11b31089d24a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.523191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb27625-5e77-46db-8d0d-11b31089d24a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.524078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb27625-5e77-46db-8d0d-11b31089d24a None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=44d5bd3f9dfe45ecb8d1a51d248f3c54) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:18.524482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb27625-5e77-46db-8d0d-11b31089d24a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:18.526057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb27625-5e77-46db-8d0d-11b31089d24a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:18.538821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 347/704] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:18 2022] GET /identity/v3/role_assignments?user.id=44d5bd3f9dfe45ecb8d1a51d248f3c54&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:18.538821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd498711-adc7-441d-9243-5200d3feb923 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.538821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd498711-adc7-441d-9243-5200d3feb923 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.538821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd498711-adc7-441d-9243-5200d3feb923 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.539846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 348/705] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:18.549297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba2f40-fe7a-46ab-93e5-9c0a34229c48 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.549634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba2f40-fe7a-46ab-93e5-9c0a34229c48 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.549913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba2f40-fe7a-46ab-93e5-9c0a34229c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.586083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2dba2f40-fe7a-46ab-93e5-9c0a34229c48 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:18.587503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2dba2f40-fe7a-46ab-93e5-9c0a34229c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:18.609164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:18.669551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a1c1-c6e6-4a04-a542-d0451b11ea03 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:18.687282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:18.720684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291647c-609f-45fc-b4eb-bbcd106bdcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:18.721551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291647c-609f-45fc-b4eb-bbcd106bdcea None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.721840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291647c-609f-45fc-b4eb-bbcd106bdcea None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.722124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291647c-609f-45fc-b4eb-bbcd106bdcea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.737013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291647c-609f-45fc-b4eb-bbcd106bdcea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:18.738591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291647c-609f-45fc-b4eb-bbcd106bdcea None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:18.748060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 349/706] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0291647c-609f-45fc-b4eb-bbcd106bdcea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:18.757598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:18.758628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:18.759745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:18.759745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:18.759745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:18.759745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:18.759745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 358/707] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:18.765805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba2f40-fe7a-46ab-93e5-9c0a34229c48 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:18.770473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291647c-609f-45fc-b4eb-bbcd106bdcea None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:18.888430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba2f40-fe7a-46ab-93e5-9c0a34229c48 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:18.895333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291647c-609f-45fc-b4eb-bbcd106bdcea None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:18.909585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:18.933143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac3d74-65ce-4813-aeb4-2c577005cf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:18.933907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac3d74-65ce-4813-aeb4-2c577005cf51 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.934189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac3d74-65ce-4813-aeb4-2c577005cf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.934462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac3d74-65ce-4813-aeb4-2c577005cf51 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.935190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac3d74-65ce-4813-aeb4-2c577005cf51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:18.935492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac3d74-65ce-4813-aeb4-2c577005cf51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:18.937408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac3d74-65ce-4813-aeb4-2c577005cf51 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:18.940234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1933fd5-f431-48d7-bc2d-3c39f0f7dfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:18.941241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1933fd5-f431-48d7-bc2d-3c39f0f7dfc3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:18.941670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1933fd5-f431-48d7-bc2d-3c39f0f7dfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:18.942111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1933fd5-f431-48d7-bc2d-3c39f0f7dfc3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:18.942914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1933fd5-f431-48d7-bc2d-3c39f0f7dfc3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:18.945039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1933fd5-f431-48d7-bc2d-3c39f0f7dfc3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:18.952535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 350/708] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:18.961708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac3d74-65ce-4813-aeb4-2c577005cf51 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a1933fd5-f431-48d7-bc2d-3c39f0f7dfc3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:18.969641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:18.970718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:18.971546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 359/709] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:18 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:18.979113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1933fd5-f431-48d7-bc2d-3c39f0f7dfc3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:18.999105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:19.088163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac3d74-65ce-4813-aeb4-2c577005cf51 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:19.124155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd2bff-3def-4ccc-989d-4c08c3d9039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:19.125128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd2bff-3def-4ccc-989d-4c08c3d9039c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.125410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd2bff-3def-4ccc-989d-4c08c3d9039c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.125778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd2bff-3def-4ccc-989d-4c08c3d9039c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.133446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd2bff-3def-4ccc-989d-4c08c3d9039c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:19.134197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1933fd5-f431-48d7-bc2d-3c39f0f7dfc3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:19.134941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd2bff-3def-4ccc-989d-4c08c3d9039c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1fbd2bff-3def-4ccc-989d-4c08c3d9039c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:19.151103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:19.152033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:19.152942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:19.152942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:19.152942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:19.152942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:19.152942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:19.152942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:19.152942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:19.152942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:19.152942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 351/710] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:19.158627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd2bff-3def-4ccc-989d-4c08c3d9039c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:19.168569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2877764-c9f9-4e5d-897c-0c7784793e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:19.169277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2877764-c9f9-4e5d-897c-0c7784793e3f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.169530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2877764-c9f9-4e5d-897c-0c7784793e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.169804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2877764-c9f9-4e5d-897c-0c7784793e3f None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.170570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2877764-c9f9-4e5d-897c-0c7784793e3f None admin] RBAC: Adding query filter params (name=network) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:19.170876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2877764-c9f9-4e5d-897c-0c7784793e3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:19.172258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2877764-c9f9-4e5d-897c-0c7784793e3f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:19.177202 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 360/711] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:19.178453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c2628-fb6a-43e1-9cdc-ae5215de1f52 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.178742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c2628-fb6a-43e1-9cdc-ae5215de1f52 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.179030 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c2628-fb6a-43e1-9cdc-ae5215de1f52 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.184774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 361/712] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:19.185637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c2628-fb6a-43e1-9cdc-ae5215de1f52 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:19.228538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:19.257938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:19.271977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd2bff-3def-4ccc-989d-4c08c3d9039c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:19.291034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c2628-fb6a-43e1-9cdc-ae5215de1f52 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:19.308770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cccbe-35b1-4ce4-b32f-767436ff2ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:19.309428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cccbe-35b1-4ce4-b32f-767436ff2ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.309710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cccbe-35b1-4ce4-b32f-767436ff2ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.309983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cccbe-35b1-4ce4-b32f-767436ff2ba8 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.310927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cccbe-35b1-4ce4-b32f-767436ff2ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:19.311236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cccbe-35b1-4ce4-b32f-767436ff2ba8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:19.312759 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cccbe-35b1-4ce4-b32f-767436ff2ba8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:19.324697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee263d20-91d6-44b4-a634-aff86d58b917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:19.325397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee263d20-91d6-44b4-a634-aff86d58b917 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.325679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee263d20-91d6-44b4-a634-aff86d58b917 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.325954 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee263d20-91d6-44b4-a634-aff86d58b917 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.326719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee263d20-91d6-44b4-a634-aff86d58b917 None admin] RBAC: Adding query filter params (type=network) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:19.327023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee263d20-91d6-44b4-a634-aff86d58b917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:19.328392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee263d20-91d6-44b4-a634-aff86d58b917 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:19.330728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 352/713] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:19.332304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf29758-833e-4f76-9357-a201dd1c736c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.332596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf29758-833e-4f76-9357-a201dd1c736c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.332898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf29758-833e-4f76-9357-a201dd1c736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.334812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 362/714] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:19.335583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee263d20-91d6-44b4-a634-aff86d58b917 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:19.351017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8cf29758-833e-4f76-9357-a201dd1c736c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:19.352240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8cf29758-833e-4f76-9357-a201dd1c736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:19.431316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 353/715] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:19.458074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf29758-833e-4f76-9357-a201dd1c736c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:19.510328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee263d20-91d6-44b4-a634-aff86d58b917 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:19.512275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:19.584875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fea4a-a852-4fa3-9c59-b5e5642511be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:19.585541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fea4a-a852-4fa3-9c59-b5e5642511be None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.585827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fea4a-a852-4fa3-9c59-b5e5642511be None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.586110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fea4a-a852-4fa3-9c59-b5e5642511be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.586958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fea4a-a852-4fa3-9c59-b5e5642511be None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=bec9fee282a64933aa0f6fbdd91a93c4) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:19.587274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fea4a-a852-4fa3-9c59-b5e5642511be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:19.588765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fea4a-a852-4fa3-9c59-b5e5642511be None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:19.595083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 363/716] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:19 2022] GET /identity/v3/role_assignments?user.id=bec9fee282a64933aa0f6fbdd91a93c4&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:19.609945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:19.638979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf29758-833e-4f76-9357-a201dd1c736c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:19.676802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b07b5a-2416-4be1-b055-d69429802538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:19.677681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07b5a-2416-4be1-b055-d69429802538 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.677959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07b5a-2416-4be1-b055-d69429802538 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.678277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07b5a-2416-4be1-b055-d69429802538 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.678946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b07b5a-2416-4be1-b055-d69429802538 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:19.680310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b07b5a-2416-4be1-b055-d69429802538 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c2b07b5a-2416-4be1-b055-d69429802538 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:19.694740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:19.695734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:19.696544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 354/717] 10.222.0.69 () {56 vars in 1129 bytes} [Fri Jun 24 10:23:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:19.702401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fea4a-a852-4fa3-9c59-b5e5642511be None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:19.822905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fea4a-a852-4fa3-9c59-b5e5642511be None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:19.860235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a3ee7-e7e3-4c78-9b60-81a61c6f3eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:19.861346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a3ee7-e7e3-4c78-9b60-81a61c6f3eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.861827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a3ee7-e7e3-4c78-9b60-81a61c6f3eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.862293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a3ee7-e7e3-4c78-9b60-81a61c6f3eb1 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.863194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a3ee7-e7e3-4c78-9b60-81a61c6f3eb1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:19.863717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a3ee7-e7e3-4c78-9b60-81a61c6f3eb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:19.865337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a3ee7-e7e3-4c78-9b60-81a61c6f3eb1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:19.876600 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 364/718] 10.222.0.69 () {56 vars in 1124 bytes} [Fri Jun 24 10:23:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:19.882256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b07b5a-2416-4be1-b055-d69429802538 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:20.023610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b07b5a-2416-4be1-b055-d69429802538 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:20.063749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e4786-d9e9-4b2d-8cc1-3bb8d3b47114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:20.064820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e4786-d9e9-4b2d-8cc1-3bb8d3b47114 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:20.065302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e4786-d9e9-4b2d-8cc1-3bb8d3b47114 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:20.065772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e4786-d9e9-4b2d-8cc1-3bb8d3b47114 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:20.066679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e4786-d9e9-4b2d-8cc1-3bb8d3b47114 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:20.067201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e4786-d9e9-4b2d-8cc1-3bb8d3b47114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:20.068790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e4786-d9e9-4b2d-8cc1-3bb8d3b47114 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:20.075076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 355/719] 10.222.0.69 () {56 vars in 1124 bytes} [Fri Jun 24 10:23:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:21.086712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8576833-890c-45ee-892c-e47490e71c5a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.087235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8576833-890c-45ee-892c-e47490e71c5a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.087349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8576833-890c-45ee-892c-e47490e71c5a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.088898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 365/720] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:21.094061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e9d2-a14d-4f19-aeb6-fc46ad8a8f46 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.094443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e9d2-a14d-4f19-aeb6-fc46ad8a8f46 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.094634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e9d2-a14d-4f19-aeb6-fc46ad8a8f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.106813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:21.112635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e028e9d2-a14d-4f19-aeb6-fc46ad8a8f46 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:21.114468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e028e9d2-a14d-4f19-aeb6-fc46ad8a8f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:21.196715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 356/721] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:21.242282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8576833-890c-45ee-892c-e47490e71c5a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:21.282728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:21.386817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8576833-890c-45ee-892c-e47490e71c5a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:21.435285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc1ae5-7e08-4d50-bd4c-64f3cbb98ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:21.436128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc1ae5-7e08-4d50-bd4c-64f3cbb98ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.436573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc1ae5-7e08-4d50-bd4c-64f3cbb98ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.436885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc1ae5-7e08-4d50-bd4c-64f3cbb98ca4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.439645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bac517-640d-41e1-a482-c28519b0a9f7 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.439936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bac517-640d-41e1-a482-c28519b0a9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.440210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bac517-640d-41e1-a482-c28519b0a9f7 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.446657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 357/722] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:21.458215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e41600-e5d0-4989-b91e-0d09899fb090 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.458702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e41600-e5d0-4989-b91e-0d09899fb090 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.458996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e41600-e5d0-4989-b91e-0d09899fb090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.467462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc1ae5-7e08-4d50-bd4c-64f3cbb98ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-20cc1ae5-7e08-4d50-bd4c-64f3cbb98ca4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:21.475247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:21.476192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:21.477068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:21.481095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d6e41600-e5d0-4989-b91e-0d09899fb090 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:21.482337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d6e41600-e5d0-4989-b91e-0d09899fb090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:21.485136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 366/723] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:21.486357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc1ae5-7e08-4d50-bd4c-64f3cbb98ca4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:21.581896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 358/724] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:23:21.609503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e41600-e5d0-4989-b91e-0d09899fb090 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:21.641901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:21.652429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc1ae5-7e08-4d50-bd4c-64f3cbb98ca4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:21.687425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cf05b-fa09-4065-9b47-e7bf660d50b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:21.688475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cf05b-fa09-4065-9b47-e7bf660d50b9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.689125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cf05b-fa09-4065-9b47-e7bf660d50b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.689602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cf05b-fa09-4065-9b47-e7bf660d50b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.690587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cf05b-fa09-4065-9b47-e7bf660d50b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:21.691101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cf05b-fa09-4065-9b47-e7bf660d50b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:21.692831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cf05b-fa09-4065-9b47-e7bf660d50b9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:21.707798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 367/725] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:21.709569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cf05b-fa09-4065-9b47-e7bf660d50b9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:21.749124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:21.821438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cf05b-fa09-4065-9b47-e7bf660d50b9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:21.834698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e41600-e5d0-4989-b91e-0d09899fb090 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:21.856925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09bb34-0c0e-4330-a9ed-48d9f5402274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:21.858228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09bb34-0c0e-4330-a9ed-48d9f5402274 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.858662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09bb34-0c0e-4330-a9ed-48d9f5402274 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.859119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09bb34-0c0e-4330-a9ed-48d9f5402274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.870416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09bb34-0c0e-4330-a9ed-48d9f5402274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:21.871884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09bb34-0c0e-4330-a9ed-48d9f5402274 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:21.882689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3ba05-0f49-48f5-9463-9d0d3e1f9165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:21.883758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3ba05-0f49-48f5-9463-9d0d3e1f9165 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.884243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3ba05-0f49-48f5-9463-9d0d3e1f9165 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.884756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3ba05-0f49-48f5-9463-9d0d3e1f9165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-2f09bb34-0c0e-4330-a9ed-48d9f5402274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:21.889799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:21.890683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:21.891561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:21.891561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:21.891561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:21.891561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:21.895671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 368/726] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:21.896983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dc5ae-51c4-4d77-9e3a-85c8f7691cdb None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.897275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dc5ae-51c4-4d77-9e3a-85c8f7691cdb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.897560 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dc5ae-51c4-4d77-9e3a-85c8f7691cdb None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.899113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 369/727] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:21.903456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dc5ae-51c4-4d77-9e3a-85c8f7691cdb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:21.903586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3ba05-0f49-48f5-9463-9d0d3e1f9165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:21.908292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2de3ba05-0f49-48f5-9463-9d0d3e1f9165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:21.909698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:21.910610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:21.911441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:21.915194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 359/728] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:21.916483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f899c-3544-46ae-a480-f88c8181ce48 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:21.916793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f899c-3544-46ae-a480-f88c8181ce48 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:21.918076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f899c-3544-46ae-a480-f88c8181ce48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:21.933880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-873f899c-3544-46ae-a480-f88c8181ce48 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:21.935067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-873f899c-3544-46ae-a480-f88c8181ce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:22.012791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 360/729] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:22.012791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f899c-3544-46ae-a480-f88c8181ce48 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:22.033493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dc5ae-51c4-4d77-9e3a-85c8f7691cdb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.069643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:22.080885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a47e2-c748-4033-a117-5048de891e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:22.081568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a47e2-c748-4033-a117-5048de891e81 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.081857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a47e2-c748-4033-a117-5048de891e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.082137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a47e2-c748-4033-a117-5048de891e81 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.082862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a47e2-c748-4033-a117-5048de891e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:22.083172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a47e2-c748-4033-a117-5048de891e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:22.084645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a47e2-c748-4033-a117-5048de891e81 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:22.097940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 370/730] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:22.100001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a47e2-c748-4033-a117-5048de891e81 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:22.117133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f899c-3544-46ae-a480-f88c8181ce48 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.156343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c50510-028a-437d-9664-28192d25db35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:22.157056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c50510-028a-437d-9664-28192d25db35 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.157343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c50510-028a-437d-9664-28192d25db35 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.157620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c50510-028a-437d-9664-28192d25db35 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.158377 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c50510-028a-437d-9664-28192d25db35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:22.158895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c50510-028a-437d-9664-28192d25db35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:22.160081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c50510-028a-437d-9664-28192d25db35 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:22.172813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 361/731] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:22.172813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732490d4-ce86-4f85-a29f-a0ae7bfb4ef7 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.172813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732490d4-ce86-4f85-a29f-a0ae7bfb4ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.172813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732490d4-ce86-4f85-a29f-a0ae7bfb4ef7 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.172813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 362/732] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:22.172813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732490d4-ce86-4f85-a29f-a0ae7bfb4ef7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:22.179988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:22.222067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a47e2-c748-4033-a117-5048de891e81 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.258634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163befeb-6e59-4b70-beb6-c20ec669c49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:22.259303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163befeb-6e59-4b70-beb6-c20ec669c49c None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.259587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163befeb-6e59-4b70-beb6-c20ec669c49c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.259864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163befeb-6e59-4b70-beb6-c20ec669c49c None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.260713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163befeb-6e59-4b70-beb6-c20ec669c49c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:22.262270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163befeb-6e59-4b70-beb6-c20ec669c49c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:22.278322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732490d4-ce86-4f85-a29f-a0ae7bfb4ef7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.288762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 371/733] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:23:22 2022] POST /identity/v3/services => generated 240 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:22.288762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163befeb-6e59-4b70-beb6-c20ec669c49c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:22.309605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:22.351527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c558d4-7e93-4db3-a2a2-4858a6536dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:22.352539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c558d4-7e93-4db3-a2a2-4858a6536dba None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.352866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c558d4-7e93-4db3-a2a2-4858a6536dba None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.353145 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c558d4-7e93-4db3-a2a2-4858a6536dba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.363499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c558d4-7e93-4db3-a2a2-4858a6536dba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:22.365784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c558d4-7e93-4db3-a2a2-4858a6536dba None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-39c558d4-7e93-4db3-a2a2-4858a6536dba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:22.387290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:22.388492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:22.390276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:22.393239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 363/734] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:23:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:22.394459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea3e2d-0333-4b67-a507-21df3b4fb8cc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.394749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea3e2d-0333-4b67-a507-21df3b4fb8cc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.395024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea3e2d-0333-4b67-a507-21df3b4fb8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.413732 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0fea3e2d-0333-4b67-a507-21df3b4fb8cc None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:22.415238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0fea3e2d-0333-4b67-a507-21df3b4fb8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:22.421591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163befeb-6e59-4b70-beb6-c20ec669c49c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.455917 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:22.477722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2bde-a4c6-4747-85ba-92055957356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:22.480048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2bde-a4c6-4747-85ba-92055957356b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.480048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2bde-a4c6-4747-85ba-92055957356b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.480048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2bde-a4c6-4747-85ba-92055957356b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.497685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d2bde-a4c6-4747-85ba-92055957356b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:22.499578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d2bde-a4c6-4747-85ba-92055957356b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:22.505395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 364/735] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:22 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:22.506253 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea3e2d-0333-4b67-a507-21df3b4fb8cc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-348d2bde-a4c6-4747-85ba-92055957356b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:22.523358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:22.524268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:22.525263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:22.525263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:22.525263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:22.525263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:22.525263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 372/736] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:22.530604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2bde-a4c6-4747-85ba-92055957356b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:22.636301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea3e2d-0333-4b67-a507-21df3b4fb8cc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.637399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2bde-a4c6-4747-85ba-92055957356b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.639018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:22.674384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ab256-4cf4-41be-97b3-37e982a712e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:22.674959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab256-4cf4-41be-97b3-37e982a712e4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.675240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab256-4cf4-41be-97b3-37e982a712e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.675515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab256-4cf4-41be-97b3-37e982a712e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.676244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab256-4cf4-41be-97b3-37e982a712e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:22.676553 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab256-4cf4-41be-97b3-37e982a712e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:22.677198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0bb86-9e3c-46f3-87a4-debbd9f9a293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:22.677872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0bb86-9e3c-46f3-87a4-debbd9f9a293 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.677979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab256-4cf4-41be-97b3-37e982a712e4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:22.678156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0bb86-9e3c-46f3-87a4-debbd9f9a293 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.678433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0bb86-9e3c-46f3-87a4-debbd9f9a293 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.679286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0bb86-9e3c-46f3-87a4-debbd9f9a293 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:22.679593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0bb86-9e3c-46f3-87a4-debbd9f9a293 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:22.681069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0bb86-9e3c-46f3-87a4-debbd9f9a293 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:22.691323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 373/737] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:22.691815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ab256-4cf4-41be-97b3-37e982a712e4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:22.705097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 365/738] 10.222.0.69 () {56 vars in 1149 bytes} [Fri Jun 24 10:23:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:22.705097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806d01c-a17b-42d2-99a2-515754059e7a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.705694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806d01c-a17b-42d2-99a2-515754059e7a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.705977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806d01c-a17b-42d2-99a2-515754059e7a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.707479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 366/739] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:22.708183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806d01c-a17b-42d2-99a2-515754059e7a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:22.731436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:22.803438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ab256-4cf4-41be-97b3-37e982a712e4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.829626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806d01c-a17b-42d2-99a2-515754059e7a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.840974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818e656-313f-4511-8df0-7866fa8fc97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:22.841876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818e656-313f-4511-8df0-7866fa8fc97e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.842296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818e656-313f-4511-8df0-7866fa8fc97e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.842574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818e656-313f-4511-8df0-7866fa8fc97e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.853198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818e656-313f-4511-8df0-7866fa8fc97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4818e656-313f-4511-8df0-7866fa8fc97e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:22.860719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:22.861627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 374/740] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:22.862836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818e656-313f-4511-8df0-7866fa8fc97e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:22.869233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f61dad-5532-42d2-8f90-9137c2b3a43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:22.870032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61dad-5532-42d2-8f90-9137c2b3a43d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.870312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61dad-5532-42d2-8f90-9137c2b3a43d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.870583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61dad-5532-42d2-8f90-9137c2b3a43d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.878530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f61dad-5532-42d2-8f90-9137c2b3a43d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:22.879954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f61dad-5532-42d2-8f90-9137c2b3a43d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c0f61dad-5532-42d2-8f90-9137c2b3a43d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:22.894555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:22.895518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:22.896429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 367/741] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:22.897049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bafa04-8607-4bef-a4ad-9b741eb75fc9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:22.897648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bafa04-8607-4bef-a4ad-9b741eb75fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:22.897648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bafa04-8607-4bef-a4ad-9b741eb75fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:22.914218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f8bafa04-8607-4bef-a4ad-9b741eb75fc9 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:22.915391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f8bafa04-8607-4bef-a4ad-9b741eb75fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:22.932352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:22.968883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:22.976497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818e656-313f-4511-8df0-7866fa8fc97e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:22.996826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 368/742] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:22 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:23:22.996826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bafa04-8607-4bef-a4ad-9b741eb75fc9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:23.026076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4d4aa-d1f4-4bdb-b6f3-bf5f5f6b4d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:23.026732 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4d4aa-d1f4-4bdb-b6f3-bf5f5f6b4d09 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:23.027017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4d4aa-d1f4-4bdb-b6f3-bf5f5f6b4d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:23.027370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4d4aa-d1f4-4bdb-b6f3-bf5f5f6b4d09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:23.097449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bafa04-8607-4bef-a4ad-9b741eb75fc9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:23.102523 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4d4aa-d1f4-4bdb-b6f3-bf5f5f6b4d09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:23.103980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4d4aa-d1f4-4bdb-b6f3-bf5f5f6b4d09 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:23.138593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8e7d2-5b5b-4db1-a5a2-77789609aed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:23.140051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8e7d2-5b5b-4db1-a5a2-77789609aed7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:23.140568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8e7d2-5b5b-4db1-a5a2-77789609aed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:23.140934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8e7d2-5b5b-4db1-a5a2-77789609aed7 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:23.141772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8e7d2-5b5b-4db1-a5a2-77789609aed7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:23.142083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8e7d2-5b5b-4db1-a5a2-77789609aed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:23.143474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8e7d2-5b5b-4db1-a5a2-77789609aed7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:23.153844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 369/743] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:23.154726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8e7d2-5b5b-4db1-a5a2-77789609aed7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:23.172314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:23.211044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:23.218486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 375/744] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:22 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:23:23.219289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4d4aa-d1f4-4bdb-b6f3-bf5f5f6b4d09 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:23.265504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8e7d2-5b5b-4db1-a5a2-77789609aed7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:23.307288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65162d-46c2-43c3-a6ec-4c95f8a9a013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:23.307968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65162d-46c2-43c3-a6ec-4c95f8a9a013 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:23.308252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65162d-46c2-43c3-a6ec-4c95f8a9a013 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:23.308529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65162d-46c2-43c3-a6ec-4c95f8a9a013 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:23.309507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65162d-46c2-43c3-a6ec-4c95f8a9a013 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:23.309822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65162d-46c2-43c3-a6ec-4c95f8a9a013 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:23.311438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65162d-46c2-43c3-a6ec-4c95f8a9a013 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:23.321034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4d4aa-d1f4-4bdb-b6f3-bf5f5f6b4d09 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:23.331207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 370/745] 10.222.0.69 () {56 vars in 1135 bytes} [Fri Jun 24 10:23:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:23.331918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65162d-46c2-43c3-a6ec-4c95f8a9a013 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:23.360883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9115a4-4b7a-46d9-8f9e-a3e927dd86d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:23.361548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9115a4-4b7a-46d9-8f9e-a3e927dd86d5 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:23.361855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9115a4-4b7a-46d9-8f9e-a3e927dd86d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:23.362092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9115a4-4b7a-46d9-8f9e-a3e927dd86d5 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:23.362954 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9115a4-4b7a-46d9-8f9e-a3e927dd86d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:23.364380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9115a4-4b7a-46d9-8f9e-a3e927dd86d5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:23.378826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 376/746] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:23:23 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:23.390755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9115a4-4b7a-46d9-8f9e-a3e927dd86d5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:23.409617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:23.458065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65162d-46c2-43c3-a6ec-4c95f8a9a013 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:23.473159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:23.503453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fa6d4-abd8-4ab7-a220-a8b84a679718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:23.504627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fa6d4-abd8-4ab7-a220-a8b84a679718 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:23.505146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fa6d4-abd8-4ab7-a220-a8b84a679718 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:23.505618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fa6d4-abd8-4ab7-a220-a8b84a679718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:23.517511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fa6d4-abd8-4ab7-a220-a8b84a679718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:23.519431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fa6d4-abd8-4ab7-a220-a8b84a679718 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:23.529299 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9115a4-4b7a-46d9-8f9e-a3e927dd86d5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e15fa6d4-abd8-4ab7-a220-a8b84a679718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:23.538853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:23.539794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:23.540772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:23.540772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:23.540772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:23.540772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:23.547777 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 371/747] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:23.547777 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fa6d4-abd8-4ab7-a220-a8b84a679718 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:23.575269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf73c8-fe11-4c47-b2ce-ed734a390dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:23.576072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf73c8-fe11-4c47-b2ce-ed734a390dbc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:23.576653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf73c8-fe11-4c47-b2ce-ed734a390dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:23.577017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf73c8-fe11-4c47-b2ce-ed734a390dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:23.589209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf73c8-fe11-4c47-b2ce-ed734a390dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:23.590674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf73c8-fe11-4c47-b2ce-ed734a390dbc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d0cf73c8-fe11-4c47-b2ce-ed734a390dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:23.609788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:23.610758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:23.611637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:23.611637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:23.611637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:23.611637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:23.611637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 377/748] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:23.612575 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf73c8-fe11-4c47-b2ce-ed734a390dbc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:23.665004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fa6d4-abd8-4ab7-a220-a8b84a679718 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:23.686884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:23.702707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb8ce6-8f0e-48c5-8202-af79fc5cfac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:23.703468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb8ce6-8f0e-48c5-8202-af79fc5cfac5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:23.703865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb8ce6-8f0e-48c5-8202-af79fc5cfac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:23.704211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb8ce6-8f0e-48c5-8202-af79fc5cfac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:23.752377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf73c8-fe11-4c47-b2ce-ed734a390dbc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:23.762121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:23.788008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb8ce6-8f0e-48c5-8202-af79fc5cfac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:23.788942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0ceea-0c1c-4b54-8550-d8341b236109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:23.789685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ceea-0c1c-4b54-8550-d8341b236109 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:23.790035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ceea-0c1c-4b54-8550-d8341b236109 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:23.790330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb8ce6-8f0e-48c5-8202-af79fc5cfac5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:23.790456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ceea-0c1c-4b54-8550-d8341b236109 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:23.791171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0ceea-0c1c-4b54-8550-d8341b236109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:23.791561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0ceea-0c1c-4b54-8550-d8341b236109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:23.793022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0ceea-0c1c-4b54-8550-d8341b236109 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:23.809886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 378/749] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:23.811038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0ceea-0c1c-4b54-8550-d8341b236109 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:23.914063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0ceea-0c1c-4b54-8550-d8341b236109 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:23.926827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 372/750] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:23:23.926827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb8ce6-8f0e-48c5-8202-af79fc5cfac5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:23.948941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:23.962155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f82176-17ad-4968-b7c2-b98ee3300884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:23.963191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f82176-17ad-4968-b7c2-b98ee3300884 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:23.963769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f82176-17ad-4968-b7c2-b98ee3300884 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:23.964277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f82176-17ad-4968-b7c2-b98ee3300884 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:23.965301 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f82176-17ad-4968-b7c2-b98ee3300884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:23.965842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f82176-17ad-4968-b7c2-b98ee3300884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:23.967877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f82176-17ad-4968-b7c2-b98ee3300884 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:23.981833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 379/751] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:23.983207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f82176-17ad-4968-b7c2-b98ee3300884 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.015902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:24.040602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb8ce6-8f0e-48c5-8202-af79fc5cfac5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.078510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef89975-2bed-4eb4-9ce8-3405b2fd3554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.079477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef89975-2bed-4eb4-9ce8-3405b2fd3554 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.079856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef89975-2bed-4eb4-9ce8-3405b2fd3554 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.080532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef89975-2bed-4eb4-9ce8-3405b2fd3554 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.088677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef89975-2bed-4eb4-9ce8-3405b2fd3554 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.090327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef89975-2bed-4eb4-9ce8-3405b2fd3554 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.098443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f82176-17ad-4968-b7c2-b98ee3300884 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-eef89975-2bed-4eb4-9ce8-3405b2fd3554 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:24.106242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:24.107232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:24.108201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:24.112386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 373/752] 10.222.0.69 () {56 vars in 1093 bytes} [Fri Jun 24 10:23:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:24.113532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef89975-2bed-4eb4-9ce8-3405b2fd3554 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.138543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0d80e-4737-4ec8-b523-c7c7c1ca8a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.140413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0d80e-4737-4ec8-b523-c7c7c1ca8a1d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.140978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0d80e-4737-4ec8-b523-c7c7c1ca8a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.141456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0d80e-4737-4ec8-b523-c7c7c1ca8a1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.153635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0d80e-4737-4ec8-b523-c7c7c1ca8a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.155518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0d80e-4737-4ec8-b523-c7c7c1ca8a1d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-41c0d80e-4737-4ec8-b523-c7c7c1ca8a1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:24.174442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:24.175399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:24.176260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:24.176260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:24.176260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:24.176260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:24.183350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 380/753] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:24.184560 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0d80e-4737-4ec8-b523-c7c7c1ca8a1d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.199214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:24.220563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef89975-2bed-4eb4-9ce8-3405b2fd3554 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.257100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e37fc7-ec72-4d06-9205-5aeb97266023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.257907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e37fc7-ec72-4d06-9205-5aeb97266023 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.258196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e37fc7-ec72-4d06-9205-5aeb97266023 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.258471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e37fc7-ec72-4d06-9205-5aeb97266023 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.266128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e37fc7-ec72-4d06-9205-5aeb97266023 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.267509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e37fc7-ec72-4d06-9205-5aeb97266023 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.277900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d1e37fc7-ec72-4d06-9205-5aeb97266023 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:24.283175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:24.284057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:24.285508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:24.285508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:24.285508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:24.285508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:24.285508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:24.285508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:24.285508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:24.285508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:24.285508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 374/754] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:24.286257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e37fc7-ec72-4d06-9205-5aeb97266023 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.302715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0d80e-4737-4ec8-b523-c7c7c1ca8a1d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.339892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7467bde-473c-49a3-a23a-8fdd27609e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.340974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7467bde-473c-49a3-a23a-8fdd27609e0f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.341479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7467bde-473c-49a3-a23a-8fdd27609e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.342261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7467bde-473c-49a3-a23a-8fdd27609e0f None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.343305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7467bde-473c-49a3-a23a-8fdd27609e0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:24.343832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7467bde-473c-49a3-a23a-8fdd27609e0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.345677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7467bde-473c-49a3-a23a-8fdd27609e0f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.365087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 381/755] 10.222.0.69 () {56 vars in 1133 bytes} [Fri Jun 24 10:23:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:24.366433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7467bde-473c-49a3-a23a-8fdd27609e0f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.396962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e37fc7-ec72-4d06-9205-5aeb97266023 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.431758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fb70e-6cb0-4034-8a9b-872c3a3a1d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.432427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fb70e-6cb0-4034-8a9b-872c3a3a1d94 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.432697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fb70e-6cb0-4034-8a9b-872c3a3a1d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.433001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fb70e-6cb0-4034-8a9b-872c3a3a1d94 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.433751 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fb70e-6cb0-4034-8a9b-872c3a3a1d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:24.434025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fb70e-6cb0-4034-8a9b-872c3a3a1d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.435377 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fb70e-6cb0-4034-8a9b-872c3a3a1d94 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.445216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:24.453639 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 375/756] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:24.453639 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fb70e-6cb0-4034-8a9b-872c3a3a1d94 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.485169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7467bde-473c-49a3-a23a-8fdd27609e0f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.521761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ba024-0337-46f5-8e06-190d139732bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.522380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ba024-0337-46f5-8e06-190d139732bf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.522675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ba024-0337-46f5-8e06-190d139732bf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.522949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ba024-0337-46f5-8e06-190d139732bf None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.523741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ba024-0337-46f5-8e06-190d139732bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:24.524058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ba024-0337-46f5-8e06-190d139732bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.525533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ba024-0337-46f5-8e06-190d139732bf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.533669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:24.543897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 382/757] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:24.545431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7708883-438e-4be5-8da0-d951f0c11a3b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.545722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7708883-438e-4be5-8da0-d951f0c11a3b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.546052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7708883-438e-4be5-8da0-d951f0c11a3b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.548785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 383/758] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:24.548785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7708883-438e-4be5-8da0-d951f0c11a3b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.570908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fb70e-6cb0-4034-8a9b-872c3a3a1d94 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.608176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472c57-2f5e-4ac8-8f7e-efbddcd0539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.609313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472c57-2f5e-4ac8-8f7e-efbddcd0539c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.609603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472c57-2f5e-4ac8-8f7e-efbddcd0539c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.609911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472c57-2f5e-4ac8-8f7e-efbddcd0539c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.620724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48472c57-2f5e-4ac8-8f7e-efbddcd0539c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.622204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48472c57-2f5e-4ac8-8f7e-efbddcd0539c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-48472c57-2f5e-4ac8-8f7e-efbddcd0539c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:24.639789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:24.640674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:24.641593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:24.641593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:24.641593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:24.641593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:24.641593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 376/759] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:24.641858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472c57-2f5e-4ac8-8f7e-efbddcd0539c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.658412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7708883-438e-4be5-8da0-d951f0c11a3b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.693923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b658e-3982-46b0-be20-e8a4c5df9c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.694736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b658e-3982-46b0-be20-e8a4c5df9c8d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.695048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b658e-3982-46b0-be20-e8a4c5df9c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.695332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b658e-3982-46b0-be20-e8a4c5df9c8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.697799 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:24.704674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b658e-3982-46b0-be20-e8a4c5df9c8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.706694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b658e-3982-46b0-be20-e8a4c5df9c8d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-085b658e-3982-46b0-be20-e8a4c5df9c8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:24.721792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:24.722699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:24.723563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:24.723563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:24.723563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:24.723563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:24.723563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:24.723563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:24.723563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:24.723563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:24.723563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 384/760] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:24.724646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832045e1-f324-4ed8-8fd2-4ce96a54e5c2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.724956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832045e1-f324-4ed8-8fd2-4ce96a54e5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.725231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832045e1-f324-4ed8-8fd2-4ce96a54e5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.744434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-832045e1-f324-4ed8-8fd2-4ce96a54e5c2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:24.745669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-832045e1-f324-4ed8-8fd2-4ce96a54e5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:24.758096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472c57-2f5e-4ac8-8f7e-efbddcd0539c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.793257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518388f-a1c9-41ad-901f-9501d7afea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.793836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518388f-a1c9-41ad-901f-9501d7afea48 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.794144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518388f-a1c9-41ad-901f-9501d7afea48 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.794428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518388f-a1c9-41ad-901f-9501d7afea48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.795265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518388f-a1c9-41ad-901f-9501d7afea48 None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=f2360d67304049278fe17c8abfe44a2b) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:24.795574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518388f-a1c9-41ad-901f-9501d7afea48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.797037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518388f-a1c9-41ad-901f-9501d7afea48 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.802613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 377/761] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:24 2022] GET /identity/v3/role_assignments?user.id=f2360d67304049278fe17c8abfe44a2b&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:24.803940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:24.805464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518388f-a1c9-41ad-901f-9501d7afea48 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.819618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 385/762] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:24.820828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832045e1-f324-4ed8-8fd2-4ce96a54e5c2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.958573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832045e1-f324-4ed8-8fd2-4ce96a54e5c2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.969303 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518388f-a1c9-41ad-901f-9501d7afea48 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.991663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.007655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede168c3-0941-424e-98a8-707fed9b2787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.008576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede168c3-0941-424e-98a8-707fed9b2787 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.009002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede168c3-0941-424e-98a8-707fed9b2787 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.009278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede168c3-0941-424e-98a8-707fed9b2787 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.010092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede168c3-0941-424e-98a8-707fed9b2787 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:25.010384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede168c3-0941-424e-98a8-707fed9b2787 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.011828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede168c3-0941-424e-98a8-707fed9b2787 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.012539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476f4b-057c-48a8-9a4b-564968633957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.013341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476f4b-057c-48a8-9a4b-564968633957 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.013714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476f4b-057c-48a8-9a4b-564968633957 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.014136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476f4b-057c-48a8-9a4b-564968633957 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.015007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476f4b-057c-48a8-9a4b-564968633957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:25.015402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476f4b-057c-48a8-9a4b-564968633957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.016939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476f4b-057c-48a8-9a4b-564968633957 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.031936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 386/763] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:25.031936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede168c3-0941-424e-98a8-707fed9b2787 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.035751 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 378/764] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:25.039249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476f4b-057c-48a8-9a4b-564968633957 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.102672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.154433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476f4b-057c-48a8-9a4b-564968633957 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.162371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede168c3-0941-424e-98a8-707fed9b2787 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.199950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab22e0-620b-495b-afaf-a5b7bcaad4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.201054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab22e0-620b-495b-afaf-a5b7bcaad4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.201543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862a64b-c1eb-426b-a642-02b6a3021a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.202010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab22e0-620b-495b-afaf-a5b7bcaad4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.202103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab22e0-620b-495b-afaf-a5b7bcaad4f2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.202850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab22e0-620b-495b-afaf-a5b7bcaad4f2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.203038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862a64b-c1eb-426b-a642-02b6a3021a43 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.203448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862a64b-c1eb-426b-a642-02b6a3021a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.203852 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862a64b-c1eb-426b-a642-02b6a3021a43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.204554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab22e0-620b-495b-afaf-a5b7bcaad4f2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.204999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862a64b-c1eb-426b-a642-02b6a3021a43 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=44d5bd3f9dfe45ecb8d1a51d248f3c54) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:25.205769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862a64b-c1eb-426b-a642-02b6a3021a43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.207946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862a64b-c1eb-426b-a642-02b6a3021a43 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-e6ab22e0-620b-495b-afaf-a5b7bcaad4f2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:25.224793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:25.225813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:25.226610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 387/765] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:25.227487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab22e0-620b-495b-afaf-a5b7bcaad4f2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.228108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 379/766] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/role_assignments?user.id=44d5bd3f9dfe45ecb8d1a51d248f3c54&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:25.229434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862a64b-c1eb-426b-a642-02b6a3021a43 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.285576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.359916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862a64b-c1eb-426b-a642-02b6a3021a43 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.406897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3e453-4a05-4421-b6c2-e31ddaee0d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.407634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3e453-4a05-4421-b6c2-e31ddaee0d1d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.407918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3e453-4a05-4421-b6c2-e31ddaee0d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.408193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3e453-4a05-4421-b6c2-e31ddaee0d1d None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.409263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3e453-4a05-4421-b6c2-e31ddaee0d1d None admin] RBAC: Adding query filter params (name=network) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:25.409574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3e453-4a05-4421-b6c2-e31ddaee0d1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.410955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3e453-4a05-4421-b6c2-e31ddaee0d1d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.418275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 380/767] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:25.426374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3e453-4a05-4421-b6c2-e31ddaee0d1d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.464396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab22e0-620b-495b-afaf-a5b7bcaad4f2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.492380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.516448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf2636-f76f-4bbf-bb26-d7a031fa29c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.517449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf2636-f76f-4bbf-bb26-d7a031fa29c9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.517834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf2636-f76f-4bbf-bb26-d7a031fa29c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.518256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf2636-f76f-4bbf-bb26-d7a031fa29c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.525513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf2636-f76f-4bbf-bb26-d7a031fa29c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.527274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf2636-f76f-4bbf-bb26-d7a031fa29c9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-e7bf2636-f76f-4bbf-bb26-d7a031fa29c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:25.542577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:25.543689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:25.544817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:25.544817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:25.544817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:25.544817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:25.544817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:25.544817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:25.544817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:25.544817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:25.551152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 388/768] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:25.552939 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1bd5a-cc5b-43ee-9d38-ca46d41b8399 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.553060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1bd5a-cc5b-43ee-9d38-ca46d41b8399 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.553457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1bd5a-cc5b-43ee-9d38-ca46d41b8399 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.554918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 389/769] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:25.556303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1bd5a-cc5b-43ee-9d38-ca46d41b8399 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.572498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3e453-4a05-4421-b6c2-e31ddaee0d1d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.576121 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.616303 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccabb7-3189-4a2b-9338-7cc08e10e330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.617339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccabb7-3189-4a2b-9338-7cc08e10e330 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.617786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccabb7-3189-4a2b-9338-7cc08e10e330 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.618217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccabb7-3189-4a2b-9338-7cc08e10e330 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.619068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccabb7-3189-4a2b-9338-7cc08e10e330 None admin] RBAC: Adding query filter params (type=network) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:25.619527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccabb7-3189-4a2b-9338-7cc08e10e330 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.621312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccabb7-3189-4a2b-9338-7cc08e10e330 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.627295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 381/770] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:25.628985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc3dc-4189-4e9c-838c-5a76a5e8a11b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.629476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc3dc-4189-4e9c-838c-5a76a5e8a11b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.629945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc3dc-4189-4e9c-838c-5a76a5e8a11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.650589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-37fdc3dc-4189-4e9c-838c-5a76a5e8a11b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:25.651809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-37fdc3dc-4189-4e9c-838c-5a76a5e8a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:25.678424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1bd5a-cc5b-43ee-9d38-ca46d41b8399 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.716540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1737244-28fd-49d2-bf72-22e326a5507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.717810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1737244-28fd-49d2-bf72-22e326a5507e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.718341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1737244-28fd-49d2-bf72-22e326a5507e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.718814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1737244-28fd-49d2-bf72-22e326a5507e None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.719873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1737244-28fd-49d2-bf72-22e326a5507e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:25.720436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1737244-28fd-49d2-bf72-22e326a5507e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.722127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 382/771] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:25.722355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1737244-28fd-49d2-bf72-22e326a5507e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.723113 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc3dc-4189-4e9c-838c-5a76a5e8a11b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.736178 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 390/772] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:25.748870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1737244-28fd-49d2-bf72-22e326a5507e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.784492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.825206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.829506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc3dc-4189-4e9c-838c-5a76a5e8a11b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.862739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1737244-28fd-49d2-bf72-22e326a5507e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.864637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8d96d-f765-43dc-afbc-f29a3219cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.865304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8d96d-f765-43dc-afbc-f29a3219cc19 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.865588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8d96d-f765-43dc-afbc-f29a3219cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.865868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8d96d-f765-43dc-afbc-f29a3219cc19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.867116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8d96d-f765-43dc-afbc-f29a3219cc19 None admin] RBAC: Adding query filter params (interface=public, service_id=2b0c23ceb233469198322f850980d65e, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:25.867116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8d96d-f765-43dc-afbc-f29a3219cc19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.868515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8d96d-f765-43dc-afbc-f29a3219cc19 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.881217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 383/773] 10.222.0.69 () {56 vars in 1251 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/endpoints?service_id=2b0c23ceb233469198322f850980d65e&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:25.882097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8d96d-f765-43dc-afbc-f29a3219cc19 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.902058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c6db5-d20d-4344-b370-f333eed554e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.903368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c6db5-d20d-4344-b370-f333eed554e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.903894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c6db5-d20d-4344-b370-f333eed554e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.904365 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c6db5-d20d-4344-b370-f333eed554e5 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/bec9fee282a64933aa0f6fbdd91a93c4/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.906732 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c6db5-d20d-4344-b370-f333eed554e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=bec9fee282a64933aa0f6fbdd91a93c4, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.908512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c6db5-d20d-4344-b370-f333eed554e5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.929156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 391/774] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:23:25 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/bec9fee282a64933aa0f6fbdd91a93c4/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:25.930471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c6db5-d20d-4344-b370-f333eed554e5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.061287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:26.073664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c6db5-d20d-4344-b370-f333eed554e5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.099272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8d96d-f765-43dc-afbc-f29a3219cc19 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.121601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f1bda-acec-417c-8c50-118ab7d0447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.122565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f1bda-acec-417c-8c50-118ab7d0447b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.122990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f1bda-acec-417c-8c50-118ab7d0447b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.123455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f1bda-acec-417c-8c50-118ab7d0447b None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.124493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f1bda-acec-417c-8c50-118ab7d0447b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.127184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f1bda-acec-417c-8c50-118ab7d0447b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.135121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 392/775] 10.222.0.69 () {56 vars in 1087 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:26.175052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7323a40-d3ac-40a7-9538-d0682e37586b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.175947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7323a40-d3ac-40a7-9538-d0682e37586b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.176230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7323a40-d3ac-40a7-9538-d0682e37586b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.176505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7323a40-d3ac-40a7-9538-d0682e37586b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.177337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7323a40-d3ac-40a7-9538-d0682e37586b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.178754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7323a40-d3ac-40a7-9538-d0682e37586b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f7323a40-d3ac-40a7-9538-d0682e37586b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:26.192445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:26.193533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:26.194885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 384/776] 10.222.0.69 () {56 vars in 1129 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:26.200265 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f1bda-acec-417c-8c50-118ab7d0447b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.323865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f1bda-acec-417c-8c50-118ab7d0447b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.374971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a47ce-6aeb-49e5-8c2b-d1f8be26a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.375564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a47ce-6aeb-49e5-8c2b-d1f8be26a928 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.375854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a47ce-6aeb-49e5-8c2b-d1f8be26a928 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.376131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a47ce-6aeb-49e5-8c2b-d1f8be26a928 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.376875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a47ce-6aeb-49e5-8c2b-d1f8be26a928 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:26.377184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a47ce-6aeb-49e5-8c2b-d1f8be26a928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.378564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a47ce-6aeb-49e5-8c2b-d1f8be26a928 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.384978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 393/777] 10.222.0.69 () {56 vars in 1124 bytes} [Fri Jun 24 10:23:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:26.391033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7323a40-d3ac-40a7-9538-d0682e37586b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.434174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:26.519823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7323a40-d3ac-40a7-9538-d0682e37586b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.556939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada7740-cd74-4f21-888b-584bdcbdc15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.557621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada7740-cd74-4f21-888b-584bdcbdc15f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.557906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada7740-cd74-4f21-888b-584bdcbdc15f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.558183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada7740-cd74-4f21-888b-584bdcbdc15f None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.558901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada7740-cd74-4f21-888b-584bdcbdc15f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:26.559210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada7740-cd74-4f21-888b-584bdcbdc15f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.560580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada7740-cd74-4f21-888b-584bdcbdc15f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.566622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 385/778] 10.222.0.69 () {56 vars in 1124 bytes} [Fri Jun 24 10:23:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:26.572384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a47ce-6aeb-49e5-8c2b-d1f8be26a928 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.610852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:26.715481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a47ce-6aeb-49e5-8c2b-d1f8be26a928 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.764042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc786b-ac2c-4684-9692-a27fcb946ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.764547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc786b-ac2c-4684-9692-a27fcb946ffd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.764854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc786b-ac2c-4684-9692-a27fcb946ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.765131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc786b-ac2c-4684-9692-a27fcb946ffd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.765938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc786b-ac2c-4684-9692-a27fcb946ffd None admin] RBAC: Adding query filter params (interface=public, service_id=e9583276ff654268b031f7f2c52ad86d, region_id=RegionOne) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:26.766246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc786b-ac2c-4684-9692-a27fcb946ffd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.767616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc786b-ac2c-4684-9692-a27fcb946ffd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.779219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 394/779] 10.222.0.69 () {56 vars in 1251 bytes} [Fri Jun 24 10:23:26 2022] GET /identity/v3/endpoints?service_id=e9583276ff654268b031f7f2c52ad86d&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:26.784449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada7740-cd74-4f21-888b-584bdcbdc15f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.938405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:26.954215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada7740-cd74-4f21-888b-584bdcbdc15f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.986500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c571d-c5c7-43d6-ac9b-0b8782fdf68c None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.986785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c571d-c5c7-43d6-ac9b-0b8782fdf68c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.986984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c571d-c5c7-43d6-ac9b-0b8782fdf68c None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.988635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 395/780] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:26.998236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e633a-6aa7-44de-afe8-d905a28528ac None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.998521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e633a-6aa7-44de-afe8-d905a28528ac None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.998808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e633a-6aa7-44de-afe8-d905a28528ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.999151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9fd20-80db-4fbc-9523-5906d03d7b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:27.000198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9fd20-80db-4fbc-9523-5906d03d7b2c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.000674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9fd20-80db-4fbc-9523-5906d03d7b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.001201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9fd20-80db-4fbc-9523-5906d03d7b2c None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.002085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9fd20-80db-4fbc-9523-5906d03d7b2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:27.003666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9fd20-80db-4fbc-9523-5906d03d7b2c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:27.010273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 386/781] 10.222.0.69 () {56 vars in 1087 bytes} [Fri Jun 24 10:23:26 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:27.039314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-122e633a-6aa7-44de-afe8-d905a28528ac None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:27.040555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-122e633a-6aa7-44de-afe8-d905a28528ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:27.149435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 396/782] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:27.184329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9fd20-80db-4fbc-9523-5906d03d7b2c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:27.311585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9fd20-80db-4fbc-9523-5906d03d7b2c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:27.351585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb257ca-f2e5-4c5e-8714-a0a873128227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:27.352402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb257ca-f2e5-4c5e-8714-a0a873128227 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.352687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb257ca-f2e5-4c5e-8714-a0a873128227 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.352994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb257ca-f2e5-4c5e-8714-a0a873128227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.365051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb257ca-f2e5-4c5e-8714-a0a873128227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4cb257ca-f2e5-4c5e-8714-a0a873128227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:27.373155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:27.374258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:27.375217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 387/783] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:27.380436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e633a-6aa7-44de-afe8-d905a28528ac None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:27.396405 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:27.507803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472ecd-6799-43a9-a850-05ca050c94c0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.508313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472ecd-6799-43a9-a850-05ca050c94c0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.508417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472ecd-6799-43a9-a850-05ca050c94c0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.509957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 388/784] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:27.516450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26688a9b-37c2-480f-8b71-d1091d1707f3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.516747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26688a9b-37c2-480f-8b71-d1091d1707f3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.517039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26688a9b-37c2-480f-8b71-d1091d1707f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.536163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-26688a9b-37c2-480f-8b71-d1091d1707f3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:27.537433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-26688a9b-37c2-480f-8b71-d1091d1707f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:27.553262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e633a-6aa7-44de-afe8-d905a28528ac None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:27.557081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:27.619765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36250458-53f8-48f3-a783-85a294925ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:27.620358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36250458-53f8-48f3-a783-85a294925ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.620634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36250458-53f8-48f3-a783-85a294925ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.620938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36250458-53f8-48f3-a783-85a294925ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.622841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36250458-53f8-48f3-a783-85a294925ce2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:27.623176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36250458-53f8-48f3-a783-85a294925ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:27.624642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36250458-53f8-48f3-a783-85a294925ce2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:27.638722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 397/785] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:27.645304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36250458-53f8-48f3-a783-85a294925ce2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:27.656997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 389/786] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:27.682322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26688a9b-37c2-480f-8b71-d1091d1707f3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:27.811857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:27.825668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26688a9b-37c2-480f-8b71-d1091d1707f3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:27.853446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36250458-53f8-48f3-a783-85a294925ce2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:27.892076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1b523-706c-40a6-8075-e4fef6686dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:27.892820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1b523-706c-40a6-8075-e4fef6686dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.893106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1b523-706c-40a6-8075-e4fef6686dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.893381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1b523-706c-40a6-8075-e4fef6686dd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.913622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1b523-706c-40a6-8075-e4fef6686dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-01c1b523-706c-40a6-8075-e4fef6686dd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:27.921374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:27.922275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:27.923107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 390/787] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:27.930079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c42225-a9d5-4dcd-afa8-786a24e43540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:27.930918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c42225-a9d5-4dcd-afa8-786a24e43540 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.931257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c42225-a9d5-4dcd-afa8-786a24e43540 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.931542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c42225-a9d5-4dcd-afa8-786a24e43540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.939224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1b523-706c-40a6-8075-e4fef6686dd6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:27.945622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c42225-a9d5-4dcd-afa8-786a24e43540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:27.947086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c42225-a9d5-4dcd-afa8-786a24e43540 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:27.948659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-15c42225-a9d5-4dcd-afa8-786a24e43540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:27.975411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:27.977424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:27.978734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:27.978734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:27.978734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:27.978734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:27.978734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 398/788] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:27.985857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c42225-a9d5-4dcd-afa8-786a24e43540 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.115904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1b523-706c-40a6-8075-e4fef6686dd6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.152854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5fb5db-7af3-4334-9a9c-bba0f91fea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.153463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fb5db-7af3-4334-9a9c-bba0f91fea53 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.153746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fb5db-7af3-4334-9a9c-bba0f91fea53 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.154021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fb5db-7af3-4334-9a9c-bba0f91fea53 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.154810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5fb5db-7af3-4334-9a9c-bba0f91fea53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:28.155121 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5fb5db-7af3-4334-9a9c-bba0f91fea53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.156499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5fb5db-7af3-4334-9a9c-bba0f91fea53 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.165745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 391/789] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:28.171314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5fb5db-7af3-4334-9a9c-bba0f91fea53 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.212594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c42225-a9d5-4dcd-afa8-786a24e43540 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.243194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:28.243504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:28.254899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70143e32-f9fa-4bc7-9361-0262cacb5416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.255571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70143e32-f9fa-4bc7-9361-0262cacb5416 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.255853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70143e32-f9fa-4bc7-9361-0262cacb5416 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.256130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70143e32-f9fa-4bc7-9361-0262cacb5416 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.257536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70143e32-f9fa-4bc7-9361-0262cacb5416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:28.257849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70143e32-f9fa-4bc7-9361-0262cacb5416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.259225 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70143e32-f9fa-4bc7-9361-0262cacb5416 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.275540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 399/790] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:28.281865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70143e32-f9fa-4bc7-9361-0262cacb5416 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.295753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5fb5db-7af3-4334-9a9c-bba0f91fea53 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.331909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ebb5d-8692-4a05-8c79-e30324334756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.332709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ebb5d-8692-4a05-8c79-e30324334756 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.333035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ebb5d-8692-4a05-8c79-e30324334756 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.333313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ebb5d-8692-4a05-8c79-e30324334756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.359399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ebb5d-8692-4a05-8c79-e30324334756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.360914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ebb5d-8692-4a05-8c79-e30324334756 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1c0ebb5d-8692-4a05-8c79-e30324334756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:28.396753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:28.397707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:28.398567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:28.398567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:28.398567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:28.398567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:28.404693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70143e32-f9fa-4bc7-9361-0262cacb5416 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.405157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 392/791] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:28.418344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ebb5d-8692-4a05-8c79-e30324334756 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.461601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6f67a-4ba8-4ec2-b781-ad9e0f5edbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.462687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6f67a-4ba8-4ec2-b781-ad9e0f5edbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.463165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6f67a-4ba8-4ec2-b781-ad9e0f5edbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.463633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6f67a-4ba8-4ec2-b781-ad9e0f5edbf5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.472120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6f67a-4ba8-4ec2-b781-ad9e0f5edbf5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.474361 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6f67a-4ba8-4ec2-b781-ad9e0f5edbf5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f0b6f67a-4ba8-4ec2-b781-ad9e0f5edbf5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:28.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:28.492075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:28.493048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:28.496218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 400/792] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:28.502257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6f67a-4ba8-4ec2-b781-ad9e0f5edbf5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.563762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:28.614008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ebb5d-8692-4a05-8c79-e30324334756 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.619596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6f67a-4ba8-4ec2-b781-ad9e0f5edbf5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.637130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:28.657211 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496d291-a73c-42e1-b84d-5b120c8ec644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.657919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5c10b-9c4c-4646-9ef9-8314aa4bb521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.658318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496d291-a73c-42e1-b84d-5b120c8ec644 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.658797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5c10b-9c4c-4646-9ef9-8314aa4bb521 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.658911 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496d291-a73c-42e1-b84d-5b120c8ec644 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.659245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5c10b-9c4c-4646-9ef9-8314aa4bb521 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.659468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496d291-a73c-42e1-b84d-5b120c8ec644 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.659705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5c10b-9c4c-4646-9ef9-8314aa4bb521 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.660571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5c10b-9c4c-4646-9ef9-8314aa4bb521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:28.660854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496d291-a73c-42e1-b84d-5b120c8ec644 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:28.661086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5c10b-9c4c-4646-9ef9-8314aa4bb521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.661482 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496d291-a73c-42e1-b84d-5b120c8ec644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.662709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5c10b-9c4c-4646-9ef9-8314aa4bb521 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.663260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496d291-a73c-42e1-b84d-5b120c8ec644 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.680816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 393/793] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:28.682549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7297-2aeb-40aa-8a38-5c7d1a7dcb6f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.682962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7297-2aeb-40aa-8a38-5c7d1a7dcb6f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.683371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7297-2aeb-40aa-8a38-5c7d1a7dcb6f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.688003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 394/794] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:28.691151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc7297-2aeb-40aa-8a38-5c7d1a7dcb6f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.693779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 401/795] 10.222.0.69 () {56 vars in 1149 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:28.695397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f5b90-709d-4bf7-a7c2-a5a26818eddc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.695875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f5b90-709d-4bf7-a7c2-a5a26818eddc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.696309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f5b90-709d-4bf7-a7c2-a5a26818eddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.715927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e89f5b90-709d-4bf7-a7c2-a5a26818eddc None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:28.717647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e89f5b90-709d-4bf7-a7c2-a5a26818eddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:28.800758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 402/796] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:28.800758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f5b90-709d-4bf7-a7c2-a5a26818eddc None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.827262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc7297-2aeb-40aa-8a38-5c7d1a7dcb6f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.828058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:28.863736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f1ab5-c46d-41c1-ae6b-2b5480bd9c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.864532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f1ab5-c46d-41c1-ae6b-2b5480bd9c98 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.864846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f1ab5-c46d-41c1-ae6b-2b5480bd9c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.865123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f1ab5-c46d-41c1-ae6b-2b5480bd9c98 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.872348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f1ab5-c46d-41c1-ae6b-2b5480bd9c98 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.873867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f1ab5-c46d-41c1-ae6b-2b5480bd9c98 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a42f1ab5-c46d-41c1-ae6b-2b5480bd9c98 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:28.888187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:28.889248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:28.890333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 395/797] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:28.891512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351d52f-7a20-4f56-a773-3fb6cd853356 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.891512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351d52f-7a20-4f56-a773-3fb6cd853356 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.891685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351d52f-7a20-4f56-a773-3fb6cd853356 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.893080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 396/798] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:28.893741 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351d52f-7a20-4f56-a773-3fb6cd853356 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.907032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f5b90-709d-4bf7-a7c2-a5a26818eddc None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.935969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:28.943391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a626c-1b94-4d88-afed-b195d310bfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.944191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a626c-1b94-4d88-afed-b195d310bfb6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.944477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a626c-1b94-4d88-afed-b195d310bfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.944771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a626c-1b94-4d88-afed-b195d310bfb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.956111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a626c-1b94-4d88-afed-b195d310bfb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.957618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a626c-1b94-4d88-afed-b195d310bfb6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-eb3a626c-1b94-4d88-afed-b195d310bfb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:28.975623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:28.976533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:28.977447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:28.977447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:28.977447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:28.977447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:28.977447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 403/799] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:28.978123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077d97a-3a0d-4351-8bd7-9f407882bec7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.978556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077d97a-3a0d-4351-8bd7-9f407882bec7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.978556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077d97a-3a0d-4351-8bd7-9f407882bec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.995171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6077d97a-3a0d-4351-8bd7-9f407882bec7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:28.996654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6077d97a-3a0d-4351-8bd7-9f407882bec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:29.006707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351d52f-7a20-4f56-a773-3fb6cd853356 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.044018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7ea46-dff4-4472-a3dc-b79b6233f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.045170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7ea46-dff4-4472-a3dc-b79b6233f03e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.045456 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7ea46-dff4-4472-a3dc-b79b6233f03e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.045761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7ea46-dff4-4472-a3dc-b79b6233f03e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.058479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7ea46-dff4-4472-a3dc-b79b6233f03e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-26f7ea46-dff4-4472-a3dc-b79b6233f03e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:29.066308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:29.067418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:29.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 397/800] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:29.068998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7ea46-dff4-4472-a3dc-b79b6233f03e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.070374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 404/801] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:29.071365 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077d97a-3a0d-4351-8bd7-9f407882bec7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.093751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:29.186157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077d97a-3a0d-4351-8bd7-9f407882bec7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.189823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:29.194290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7ea46-dff4-4472-a3dc-b79b6233f03e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.223301 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d164d91-aa65-44ad-aa9e-3e78a34fd1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.223972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d164d91-aa65-44ad-aa9e-3e78a34fd1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.224254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d164d91-aa65-44ad-aa9e-3e78a34fd1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.224530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d164d91-aa65-44ad-aa9e-3e78a34fd1b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.225296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d164d91-aa65-44ad-aa9e-3e78a34fd1b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:29.225601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d164d91-aa65-44ad-aa9e-3e78a34fd1b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.226978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d164d91-aa65-44ad-aa9e-3e78a34fd1b5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.235828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b545-dd9f-4c8d-980f-91b41181933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.236504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b545-dd9f-4c8d-980f-91b41181933f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.236809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b545-dd9f-4c8d-980f-91b41181933f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.237088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b545-dd9f-4c8d-980f-91b41181933f None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.237927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b545-dd9f-4c8d-980f-91b41181933f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:29.242929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b545-dd9f-4c8d-980f-91b41181933f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.243061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 405/802] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:29.243061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d164d91-aa65-44ad-aa9e-3e78a34fd1b5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.243240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b545-dd9f-4c8d-980f-91b41181933f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.265136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 398/803] 10.222.0.69 () {56 vars in 1135 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:29.266300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b545-dd9f-4c8d-980f-91b41181933f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.325290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:29.349356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d164d91-aa65-44ad-aa9e-3e78a34fd1b5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.374575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b545-dd9f-4c8d-980f-91b41181933f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.387675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127f42b-777c-4eb4-989b-e2a2a06cf485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.388360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127f42b-777c-4eb4-989b-e2a2a06cf485 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.388646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127f42b-777c-4eb4-989b-e2a2a06cf485 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.389119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127f42b-777c-4eb4-989b-e2a2a06cf485 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.389973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127f42b-777c-4eb4-989b-e2a2a06cf485 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:29.390286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127f42b-777c-4eb4-989b-e2a2a06cf485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.391703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127f42b-777c-4eb4-989b-e2a2a06cf485 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.399955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 406/804] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:29.400761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127f42b-777c-4eb4-989b-e2a2a06cf485 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.414941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175a33a-ceae-46be-9f33-e2d70f510f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.416027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175a33a-ceae-46be-9f33-e2d70f510f8c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.416475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175a33a-ceae-46be-9f33-e2d70f510f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.416962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175a33a-ceae-46be-9f33-e2d70f510f8c None admin] PATH_INFO: `/v3/services/network` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.417819 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175a33a-ceae-46be-9f33-e2d70f510f8c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.419184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175a33a-ceae-46be-9f33-e2d70f510f8c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d175a33a-ceae-46be-9f33-e2d70f510f8c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:29.430722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:29.431681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:29.432533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 399/805] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:29.433827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175a33a-ceae-46be-9f33-e2d70f510f8c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.462409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:29.526462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127f42b-777c-4eb4-989b-e2a2a06cf485 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.544595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175a33a-ceae-46be-9f33-e2d70f510f8c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.561989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7f95e-bcf1-4631-b841-55cb1b1ea09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.562832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7f95e-bcf1-4631-b841-55cb1b1ea09b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.563132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7f95e-bcf1-4631-b841-55cb1b1ea09b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.563410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7f95e-bcf1-4631-b841-55cb1b1ea09b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.570750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7f95e-bcf1-4631-b841-55cb1b1ea09b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.572322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7f95e-bcf1-4631-b841-55cb1b1ea09b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.574831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:29.578044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cb251-dd53-45e3-a80f-307e69569b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.578850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cb251-dd53-45e3-a80f-307e69569b5a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.579131 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cb251-dd53-45e3-a80f-307e69569b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.579405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cb251-dd53-45e3-a80f-307e69569b5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d5c7f95e-bcf1-4631-b841-55cb1b1ea09b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:29.589706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:29.590908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 407/806] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:29.592242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7f95e-bcf1-4631-b841-55cb1b1ea09b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.593958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9cb251-dd53-45e3-a80f-307e69569b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.595516 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9cb251-dd53-45e3-a80f-307e69569b5a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6a9cb251-dd53-45e3-a80f-307e69569b5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:29.613133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:29.614631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:29.615808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:29.615808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:29.615808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:29.615808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:29.615808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 400/807] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:29.615808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cb251-dd53-45e3-a80f-307e69569b5a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.685350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:29.692984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7f95e-bcf1-4631-b841-55cb1b1ea09b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.716130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cb251-dd53-45e3-a80f-307e69569b5a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.727665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0ea4f-c952-473e-8353-24e6f8604b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.728452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0ea4f-c952-473e-8353-24e6f8604b9b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.728757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0ea4f-c952-473e-8353-24e6f8604b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.729037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0ea4f-c952-473e-8353-24e6f8604b9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.739060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0ea4f-c952-473e-8353-24e6f8604b9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.740503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0ea4f-c952-473e-8353-24e6f8604b9b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.749447 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834409a-dd85-4f4c-ba8b-8d21833dc96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.750105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834409a-dd85-4f4c-ba8b-8d21833dc96d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.750388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834409a-dd85-4f4c-ba8b-8d21833dc96d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.750659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834409a-dd85-4f4c-ba8b-8d21833dc96d None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.751367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834409a-dd85-4f4c-ba8b-8d21833dc96d None admin] RBAC: Adding query filter params (name=network) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:29.751671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834409a-dd85-4f4c-ba8b-8d21833dc96d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.753042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834409a-dd85-4f4c-ba8b-8d21833dc96d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-29a0ea4f-c952-473e-8353-24e6f8604b9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:29.757311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:29.758141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:29.764709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:29.764709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:29.764709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:29.764709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:29.764964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 401/808] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:29.764964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71a83c-47ac-4124-b379-3e9852ad11eb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.765090 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 408/809] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:29.765169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71a83c-47ac-4124-b379-3e9852ad11eb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.765169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71a83c-47ac-4124-b379-3e9852ad11eb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.765291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0ea4f-c952-473e-8353-24e6f8604b9b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.765356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 402/810] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:29.765356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71a83c-47ac-4124-b379-3e9852ad11eb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.812945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:29.862909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0ea4f-c952-473e-8353-24e6f8604b9b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.865693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71a83c-47ac-4124-b379-3e9852ad11eb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.896628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d365a-89b3-44a0-a4e2-89c6901f1604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.897310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d365a-89b3-44a0-a4e2-89c6901f1604 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.897591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d365a-89b3-44a0-a4e2-89c6901f1604 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.897861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d365a-89b3-44a0-a4e2-89c6901f1604 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.898647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d365a-89b3-44a0-a4e2-89c6901f1604 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:29.898739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fec193-0111-4b15-a9c5-a99cdb882572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.898969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d365a-89b3-44a0-a4e2-89c6901f1604 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.899345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fec193-0111-4b15-a9c5-a99cdb882572 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.899630 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fec193-0111-4b15-a9c5-a99cdb882572 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.899906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fec193-0111-4b15-a9c5-a99cdb882572 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.900400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d365a-89b3-44a0-a4e2-89c6901f1604 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.900638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fec193-0111-4b15-a9c5-a99cdb882572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:29.900963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fec193-0111-4b15-a9c5-a99cdb882572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.902332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fec193-0111-4b15-a9c5-a99cdb882572 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.904650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:29.912565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 409/811] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:29.913451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d365a-89b3-44a0-a4e2-89c6901f1604 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.919718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 403/812] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:29.919718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fec193-0111-4b15-a9c5-a99cdb882572 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.014276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d365a-89b3-44a0-a4e2-89c6901f1604 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.017337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fec193-0111-4b15-a9c5-a99cdb882572 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.027018 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:30.049566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8dfa82-0521-4930-b44a-d7cd3e795ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.050239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dfa82-0521-4930-b44a-d7cd3e795ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.050438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348ac68-5c35-4744-a95a-2dccd43ece42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.050547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dfa82-0521-4930-b44a-d7cd3e795ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.050795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dfa82-0521-4930-b44a-d7cd3e795ed8 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.051095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348ac68-5c35-4744-a95a-2dccd43ece42 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.051372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348ac68-5c35-4744-a95a-2dccd43ece42 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.051506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8dfa82-0521-4930-b44a-d7cd3e795ed8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:30.051641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348ac68-5c35-4744-a95a-2dccd43ece42 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.051817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8dfa82-0521-4930-b44a-d7cd3e795ed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.052552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348ac68-5c35-4744-a95a-2dccd43ece42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:30.052892 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348ac68-5c35-4744-a95a-2dccd43ece42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.053244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8dfa82-0521-4930-b44a-d7cd3e795ed8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.054233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348ac68-5c35-4744-a95a-2dccd43ece42 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.058936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 410/813] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:30.060230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e1fb2-e0ed-403d-ba35-47fdc360aaf7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.060526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e1fb2-e0ed-403d-ba35-47fdc360aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.060904 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e1fb2-e0ed-403d-ba35-47fdc360aaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.066117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 404/814] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:30.066901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348ac68-5c35-4744-a95a-2dccd43ece42 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.079207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9f2e1fb2-e0ed-403d-ba35-47fdc360aaf7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:30.080726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9f2e1fb2-e0ed-403d-ba35-47fdc360aaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:30.127383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:30.147505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 411/815] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:30.148427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e1fb2-e0ed-403d-ba35-47fdc360aaf7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.174825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348ac68-5c35-4744-a95a-2dccd43ece42 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.210469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b10ca-1197-44ec-beac-1a851622d5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.211357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b10ca-1197-44ec-beac-1a851622d5c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.211634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b10ca-1197-44ec-beac-1a851622d5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.211902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b10ca-1197-44ec-beac-1a851622d5c3 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/f2360d67304049278fe17c8abfe44a2b/roles/6b7e9f2fcda040b18213464589952d67` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.226085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b10ca-1197-44ec-beac-1a851622d5c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=f2360d67304049278fe17c8abfe44a2b, role_id=6b7e9f2fcda040b18213464589952d67)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.227517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b10ca-1197-44ec-beac-1a851622d5c3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.241108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 405/816] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:23:30 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/f2360d67304049278fe17c8abfe44a2b/roles/6b7e9f2fcda040b18213464589952d67 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:30.241877 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b10ca-1197-44ec-beac-1a851622d5c3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.247702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e1fb2-e0ed-403d-ba35-47fdc360aaf7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.256548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:30.290000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ef32e-e585-4d41-9eaa-bb9f05aeae70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.290790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ef32e-e585-4d41-9eaa-bb9f05aeae70 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.291068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ef32e-e585-4d41-9eaa-bb9f05aeae70 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.291346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ef32e-e585-4d41-9eaa-bb9f05aeae70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.299916 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ef32e-e585-4d41-9eaa-bb9f05aeae70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.301446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ef32e-e585-4d41-9eaa-bb9f05aeae70 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-257ef32e-e585-4d41-9eaa-bb9f05aeae70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:30.316657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:30.317636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:30.318537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:30.318537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:30.318537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:30.318537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:30.318537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:30.318537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:30.318537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:30.318537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:30.318537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 412/817] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:30.319015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ef32e-e585-4d41-9eaa-bb9f05aeae70 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.375359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b10ca-1197-44ec-beac-1a851622d5c3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.383528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:30.417486 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e0ff5-24b4-4951-9231-ae5e86015364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.418994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e0ff5-24b4-4951-9231-ae5e86015364 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.418994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e0ff5-24b4-4951-9231-ae5e86015364 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.418994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e0ff5-24b4-4951-9231-ae5e86015364 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.419478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e0ff5-24b4-4951-9231-ae5e86015364 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.421031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e0ff5-24b4-4951-9231-ae5e86015364 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.448232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 406/818] 10.222.0.69 () {60 vars in 1144 bytes} [Fri Jun 24 10:23:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:30.449560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e0ff5-24b4-4951-9231-ae5e86015364 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.565446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ef32e-e585-4d41-9eaa-bb9f05aeae70 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.591475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e0ff5-24b4-4951-9231-ae5e86015364 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.613132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202243d-1106-4c75-9fff-78b363c88164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.613945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202243d-1106-4c75-9fff-78b363c88164 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.614228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202243d-1106-4c75-9fff-78b363c88164 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.614505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202243d-1106-4c75-9fff-78b363c88164 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.623418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202243d-1106-4c75-9fff-78b363c88164 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.624993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202243d-1106-4c75-9fff-78b363c88164 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.633279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf8805-4aad-486a-aef7-a2da6f43727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.634107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf8805-4aad-486a-aef7-a2da6f43727c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.634388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf8805-4aad-486a-aef7-a2da6f43727c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.634664 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf8805-4aad-486a-aef7-a2da6f43727c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.635329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf8805-4aad-486a-aef7-a2da6f43727c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.636709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf8805-4aad-486a-aef7-a2da6f43727c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-1202243d-1106-4c75-9fff-78b363c88164 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:30.644522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:30.645620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:30.646655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:30.650462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 413/819] 10.222.0.69 () {56 vars in 1093 bytes} [Fri Jun 24 10:23:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:30.650462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202243d-1106-4c75-9fff-78b363c88164 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7acf8805-4aad-486a-aef7-a2da6f43727c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:30.650667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:30.651489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:30.652330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 407/820] 10.222.0.69 () {56 vars in 1129 bytes} [Fri Jun 24 10:23:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:30.659453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf8805-4aad-486a-aef7-a2da6f43727c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.684402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:30.688332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:30.770893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202243d-1106-4c75-9fff-78b363c88164 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.786871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf8805-4aad-486a-aef7-a2da6f43727c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.807092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ec1d4-2f18-4b82-b1b8-c2ffd1fd8652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.807872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ec1d4-2f18-4b82-b1b8-c2ffd1fd8652 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.808249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ec1d4-2f18-4b82-b1b8-c2ffd1fd8652 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.808627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ec1d4-2f18-4b82-b1b8-c2ffd1fd8652 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.809533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ec1d4-2f18-4b82-b1b8-c2ffd1fd8652 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:30.809989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ec1d4-2f18-4b82-b1b8-c2ffd1fd8652 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.811637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ec1d4-2f18-4b82-b1b8-c2ffd1fd8652 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.824115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 414/821] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:30.824974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ec1d4-2f18-4b82-b1b8-c2ffd1fd8652 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.830298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d9dab-8f0a-4997-b43d-21b53572b403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.831299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d9dab-8f0a-4997-b43d-21b53572b403 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.831581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d9dab-8f0a-4997-b43d-21b53572b403 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.831856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d9dab-8f0a-4997-b43d-21b53572b403 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.832710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d9dab-8f0a-4997-b43d-21b53572b403 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:30.833136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d9dab-8f0a-4997-b43d-21b53572b403 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.834582 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d9dab-8f0a-4997-b43d-21b53572b403 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.854871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 408/822] 10.222.0.69 () {56 vars in 1133 bytes} [Fri Jun 24 10:23:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:30.855681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d9dab-8f0a-4997-b43d-21b53572b403 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:30.923285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:30.938165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ec1d4-2f18-4b82-b1b8-c2ffd1fd8652 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.941465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:30.962646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d9dab-8f0a-4997-b43d-21b53572b403 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:30.977292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba2820-c1d2-4c81-97db-ffa32edfd360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:30.978056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba2820-c1d2-4c81-97db-ffa32edfd360 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:30.978411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba2820-c1d2-4c81-97db-ffa32edfd360 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:30.978768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba2820-c1d2-4c81-97db-ffa32edfd360 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:30.979484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba2820-c1d2-4c81-97db-ffa32edfd360 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:30.979796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba2820-c1d2-4c81-97db-ffa32edfd360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:30.981201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba2820-c1d2-4c81-97db-ffa32edfd360 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:30.986412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 415/823] 10.222.0.69 () {56 vars in 1124 bytes} [Fri Jun 24 10:23:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:30.987208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba2820-c1d2-4c81-97db-ffa32edfd360 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:31.002797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5e5e9-47b9-49a1-bb26-e49f41b752b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:31.004028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5e5e9-47b9-49a1-bb26-e49f41b752b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:31.004349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5e5e9-47b9-49a1-bb26-e49f41b752b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:31.004685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5e5e9-47b9-49a1-bb26-e49f41b752b4 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/44d5bd3f9dfe45ecb8d1a51d248f3c54/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:31.006737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5e5e9-47b9-49a1-bb26-e49f41b752b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=44d5bd3f9dfe45ecb8d1a51d248f3c54, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:31.008250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5e5e9-47b9-49a1-bb26-e49f41b752b4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:31.022339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 409/824] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:23:30 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/44d5bd3f9dfe45ecb8d1a51d248f3c54/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:31.023193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5e5e9-47b9-49a1-bb26-e49f41b752b4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:31.115529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba2820-c1d2-4c81-97db-ffa32edfd360 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:31.157940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b132bbe-4f58-401f-bb3b-67c277375a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:31.158198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5e5e9-47b9-49a1-bb26-e49f41b752b4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:31.158995 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b132bbe-4f58-401f-bb3b-67c277375a2b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:31.159423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b132bbe-4f58-401f-bb3b-67c277375a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:31.159855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b132bbe-4f58-401f-bb3b-67c277375a2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:31.184137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:31.205002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5460184-2f67-46ae-bf60-a9aa2bc4e9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:31.206442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5460184-2f67-46ae-bf60-a9aa2bc4e9ce None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:31.206872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5460184-2f67-46ae-bf60-a9aa2bc4e9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:31.207378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5460184-2f67-46ae-bf60-a9aa2bc4e9ce None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:31.208256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5460184-2f67-46ae-bf60-a9aa2bc4e9ce None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:31.208708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5460184-2f67-46ae-bf60-a9aa2bc4e9ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:31.210251 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5460184-2f67-46ae-bf60-a9aa2bc4e9ce None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:31.216590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 410/825] 10.222.0.69 () {56 vars in 1124 bytes} [Fri Jun 24 10:23:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:31.222820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5460184-2f67-46ae-bf60-a9aa2bc4e9ce None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:31.227643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:31.249926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b132bbe-4f58-401f-bb3b-67c277375a2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:31.251416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b132bbe-4f58-401f-bb3b-67c277375a2b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:31.337775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5460184-2f67-46ae-bf60-a9aa2bc4e9ce None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:31.381589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2edd3d-9f4c-4661-910c-f8b5e7543312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:31.382412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2edd3d-9f4c-4661-910c-f8b5e7543312 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:31.382707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2edd3d-9f4c-4661-910c-f8b5e7543312 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:31.383361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2edd3d-9f4c-4661-910c-f8b5e7543312 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:31.384375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2edd3d-9f4c-4661-910c-f8b5e7543312 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:31.385865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2edd3d-9f4c-4661-910c-f8b5e7543312 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:31.386842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 416/826] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:23:31.393737 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b132bbe-4f58-401f-bb3b-67c277375a2b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:31.410031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 411/827] 10.222.0.69 () {60 vars in 1144 bytes} [Fri Jun 24 10:23:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:31.462734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:31.518672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b132bbe-4f58-401f-bb3b-67c277375a2b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:31.563135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b9075-cb87-43b6-86fb-b22185ae81ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:31.563945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b9075-cb87-43b6-86fb-b22185ae81ac None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:31.564223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b9075-cb87-43b6-86fb-b22185ae81ac None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:31.564495 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b9075-cb87-43b6-86fb-b22185ae81ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:31.575680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b9075-cb87-43b6-86fb-b22185ae81ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:31.577204 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b9075-cb87-43b6-86fb-b22185ae81ac None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-690b9075-cb87-43b6-86fb-b22185ae81ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:31.596184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:31.597167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:31.598053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:31.598053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:31.598053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:31.598053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:31.598053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 417/828] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:31.602932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2edd3d-9f4c-4661-910c-f8b5e7543312 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:31.668020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:31.712637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2edd3d-9f4c-4661-910c-f8b5e7543312 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:31.747480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad1a9a-a479-4886-90da-dbff8ecfa17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:31.748223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad1a9a-a479-4886-90da-dbff8ecfa17f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:31.748518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad1a9a-a479-4886-90da-dbff8ecfa17f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:31.748821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad1a9a-a479-4886-90da-dbff8ecfa17f None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:31.749609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad1a9a-a479-4886-90da-dbff8ecfa17f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:31.749920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad1a9a-a479-4886-90da-dbff8ecfa17f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:31.751297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad1a9a-a479-4886-90da-dbff8ecfa17f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:31.763733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 412/829] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:31.769279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b9075-cb87-43b6-86fb-b22185ae81ac None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:31.875297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:31.879537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b9075-cb87-43b6-86fb-b22185ae81ac None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:31.917040 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c563c30-7d50-4e00-aa9e-cec9296416ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:31.918262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c563c30-7d50-4e00-aa9e-cec9296416ba None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:31.918776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c563c30-7d50-4e00-aa9e-cec9296416ba None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:31.919241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c563c30-7d50-4e00-aa9e-cec9296416ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:31.930377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c563c30-7d50-4e00-aa9e-cec9296416ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:31.931831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c563c30-7d50-4e00-aa9e-cec9296416ba None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7c563c30-7d50-4e00-aa9e-cec9296416ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:31.945787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:31.946755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:31.947610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:31.947610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:31.947610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:31.947610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:31.947610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:31.947610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:31.947610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:31.947610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:31.947610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 418/830] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:31.953074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad1a9a-a479-4886-90da-dbff8ecfa17f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:32.053647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad1a9a-a479-4886-90da-dbff8ecfa17f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:32.074645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:32.089137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a909a-6449-4384-a968-0e446970b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:32.090057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a909a-6449-4384-a968-0e446970b009 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.090562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a909a-6449-4384-a968-0e446970b009 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.090973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a909a-6449-4384-a968-0e446970b009 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.091949 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a909a-6449-4384-a968-0e446970b009 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:32.092404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a909a-6449-4384-a968-0e446970b009 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:32.094009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a909a-6449-4384-a968-0e446970b009 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:32.106252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 413/831] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:32.113832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c563c30-7d50-4e00-aa9e-cec9296416ba None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:32.218485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c563c30-7d50-4e00-aa9e-cec9296416ba None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:32.254064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc64cd1-337e-4bec-9646-767c30d8adac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:32.254744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc64cd1-337e-4bec-9646-767c30d8adac None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.255024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc64cd1-337e-4bec-9646-767c30d8adac None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.255296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc64cd1-337e-4bec-9646-767c30d8adac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.256154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc64cd1-337e-4bec-9646-767c30d8adac None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=bec9fee282a64933aa0f6fbdd91a93c4) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:32.256469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc64cd1-337e-4bec-9646-767c30d8adac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:32.257991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc64cd1-337e-4bec-9646-767c30d8adac None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:32.265379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 419/832] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:32 2022] GET /identity/v3/role_assignments?user.id=bec9fee282a64933aa0f6fbdd91a93c4&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:32.323158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9ad7a-8f27-4062-b3b3-b55b4f3bb9dc None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.323488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9ad7a-8f27-4062-b3b3-b55b4f3bb9dc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.323718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9ad7a-8f27-4062-b3b3-b55b4f3bb9dc None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.325520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 414/833] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:32.331750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f379d9-2915-4b46-8c65-ef9a45dc2154 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.332039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f379d9-2915-4b46-8c65-ef9a45dc2154 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.332324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f379d9-2915-4b46-8c65-ef9a45dc2154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.350463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c6f379d9-2915-4b46-8c65-ef9a45dc2154 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:32.351752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c6f379d9-2915-4b46-8c65-ef9a45dc2154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:32.370594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:32.426985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 420/834] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:32.451095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9ad7a-8f27-4062-b3b3-b55b4f3bb9dc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:32.563766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9ad7a-8f27-4062-b3b3-b55b4f3bb9dc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:32.599413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420f823-af49-4114-a80a-60749a90f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:32.600220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420f823-af49-4114-a80a-60749a90f2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.600510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420f823-af49-4114-a80a-60749a90f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.600801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420f823-af49-4114-a80a-60749a90f2e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.615607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420f823-af49-4114-a80a-60749a90f2e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4420f823-af49-4114-a80a-60749a90f2e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:32.623776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:32.624809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:32.625652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 415/835] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:23:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:32.632801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f379d9-2915-4b46-8c65-ef9a45dc2154 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:32.734501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f379d9-2915-4b46-8c65-ef9a45dc2154 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:32.773235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872502-1b63-4e05-9971-e799e3051e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:32.773908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872502-1b63-4e05-9971-e799e3051e09 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.774190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872502-1b63-4e05-9971-e799e3051e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.774461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872502-1b63-4e05-9971-e799e3051e09 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.775209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872502-1b63-4e05-9971-e799e3051e09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:32.775508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872502-1b63-4e05-9971-e799e3051e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:32.776903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872502-1b63-4e05-9971-e799e3051e09 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:32.785622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 421/836] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:23:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:32.791029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420f823-af49-4114-a80a-60749a90f2e1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:32.839603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:32.899183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420f823-af49-4114-a80a-60749a90f2e1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:32.934454 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ba2c3-d52a-4524-98ac-a77d48ae6099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:32.935855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ba2c3-d52a-4524-98ac-a77d48ae6099 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.936330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ba2c3-d52a-4524-98ac-a77d48ae6099 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.936884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ba2c3-d52a-4524-98ac-a77d48ae6099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.948729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ba2c3-d52a-4524-98ac-a77d48ae6099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:32.950236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ba2c3-d52a-4524-98ac-a77d48ae6099 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7f2ba2c3-d52a-4524-98ac-a77d48ae6099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:32.968430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:32.969335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:32.970210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:32.970210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:32.970210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:32.970210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:32.970210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 416/837] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:32.976269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872502-1b63-4e05-9971-e799e3051e09 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.005137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:33.080871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872502-1b63-4e05-9971-e799e3051e09 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.114570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53729759-5bbf-44e9-9ca7-606a0840263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.115225 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53729759-5bbf-44e9-9ca7-606a0840263c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.115503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53729759-5bbf-44e9-9ca7-606a0840263c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.115777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53729759-5bbf-44e9-9ca7-606a0840263c None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.116551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53729759-5bbf-44e9-9ca7-606a0840263c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:33.117013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53729759-5bbf-44e9-9ca7-606a0840263c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.118437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53729759-5bbf-44e9-9ca7-606a0840263c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.131066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 422/838] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:33.136495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ba2c3-d52a-4524-98ac-a77d48ae6099 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.229781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:33.238896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ba2c3-d52a-4524-98ac-a77d48ae6099 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.239055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30072fa-417f-48eb-be69-c61feb5793a2 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.239165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30072fa-417f-48eb-be69-c61feb5793a2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.239449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30072fa-417f-48eb-be69-c61feb5793a2 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.241092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 423/839] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:33.245986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed528dfc-1713-4247-ad36-914bfa3af038 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.246258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed528dfc-1713-4247-ad36-914bfa3af038 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.246533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed528dfc-1713-4247-ad36-914bfa3af038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.263414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ed528dfc-1713-4247-ad36-914bfa3af038 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:33.264788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ed528dfc-1713-4247-ad36-914bfa3af038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:33.273744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21165251-59bb-439c-ac6b-a1c410a9d11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.274564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21165251-59bb-439c-ac6b-a1c410a9d11a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.274841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21165251-59bb-439c-ac6b-a1c410a9d11a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.275119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21165251-59bb-439c-ac6b-a1c410a9d11a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.283612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21165251-59bb-439c-ac6b-a1c410a9d11a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.285196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21165251-59bb-439c-ac6b-a1c410a9d11a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-21165251-59bb-439c-ac6b-a1c410a9d11a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:33.300597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:33.301738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:33.302781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 417/840] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:33.307369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21165251-59bb-439c-ac6b-a1c410a9d11a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.349098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 424/841] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:33.366970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed528dfc-1713-4247-ad36-914bfa3af038 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.371355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:33.424876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21165251-59bb-439c-ac6b-a1c410a9d11a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.467715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66c9df-401e-4919-b2f0-3a0734dab00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.468515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c9df-401e-4919-b2f0-3a0734dab00d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.468918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c9df-401e-4919-b2f0-3a0734dab00d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.469288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c9df-401e-4919-b2f0-3a0734dab00d None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.470229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66c9df-401e-4919-b2f0-3a0734dab00d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:33.470632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66c9df-401e-4919-b2f0-3a0734dab00d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.472969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66c9df-401e-4919-b2f0-3a0734dab00d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.487192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed528dfc-1713-4247-ad36-914bfa3af038 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.496586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 418/842] 10.222.0.69 () {56 vars in 1149 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:33.502108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66c9df-401e-4919-b2f0-3a0734dab00d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.520675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:33.539305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb206b-4cb3-4d97-9e0c-b46657cfea80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.540672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb206b-4cb3-4d97-9e0c-b46657cfea80 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.541378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb206b-4cb3-4d97-9e0c-b46657cfea80 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.542218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb206b-4cb3-4d97-9e0c-b46657cfea80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.556890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb206b-4cb3-4d97-9e0c-b46657cfea80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-20cb206b-4cb3-4d97-9e0c-b46657cfea80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:33.567439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:33.568378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:23:33.569252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:33.572785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 425/843] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:33.575965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb206b-4cb3-4d97-9e0c-b46657cfea80 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.626937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66c9df-401e-4919-b2f0-3a0734dab00d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.641983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:33.660757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3e250-a53f-4eae-b0b5-24755d072611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.661404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3e250-a53f-4eae-b0b5-24755d072611 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.661684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3e250-a53f-4eae-b0b5-24755d072611 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.661951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3e250-a53f-4eae-b0b5-24755d072611 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.685208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb206b-4cb3-4d97-9e0c-b46657cfea80 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.722262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f1a0a-98c5-4270-9e30-944086338f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.723132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f1a0a-98c5-4270-9e30-944086338f87 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.723560 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f1a0a-98c5-4270-9e30-944086338f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.724012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f1a0a-98c5-4270-9e30-944086338f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.725030 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f1a0a-98c5-4270-9e30-944086338f87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:33.725477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f1a0a-98c5-4270-9e30-944086338f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.727120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f1a0a-98c5-4270-9e30-944086338f87 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.728176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3e250-a53f-4eae-b0b5-24755d072611 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.729831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3e250-a53f-4eae-b0b5-24755d072611 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.740342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 426/844] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:33.741648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:33.742263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5335b-5fbf-439b-96ff-8cf0441d3827 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.742658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5335b-5fbf-439b-96ff-8cf0441d3827 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.742821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5335b-5fbf-439b-96ff-8cf0441d3827 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.744168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 427/845] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:33.744807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5335b-5fbf-439b-96ff-8cf0441d3827 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.843733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 419/846] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:23:33.843733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c933ee4-890d-4228-a761-ce9caa9b13df None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.843733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c933ee4-890d-4228-a761-ce9caa9b13df None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.843733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c933ee4-890d-4228-a761-ce9caa9b13df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.849657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5335b-5fbf-439b-96ff-8cf0441d3827 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.860030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8c933ee4-890d-4228-a761-ce9caa9b13df None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:33.861475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8c933ee4-890d-4228-a761-ce9caa9b13df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:33.877542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:33.886775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b459d60-6cd5-4b15-8169-b27ea7a32d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.887572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b459d60-6cd5-4b15-8169-b27ea7a32d73 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.887850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b459d60-6cd5-4b15-8169-b27ea7a32d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.888120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b459d60-6cd5-4b15-8169-b27ea7a32d73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.898764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b459d60-6cd5-4b15-8169-b27ea7a32d73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.900194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b459d60-6cd5-4b15-8169-b27ea7a32d73 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-2b459d60-6cd5-4b15-8169-b27ea7a32d73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:33.917422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:33.918438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:33.919504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:33.919504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:33.919504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:33.919504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:33.923444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 428/847] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:33.924329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b459d60-6cd5-4b15-8169-b27ea7a32d73 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.939117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 420/848] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:33.940340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c933ee4-890d-4228-a761-ce9caa9b13df None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.993552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:34.034515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b459d60-6cd5-4b15-8169-b27ea7a32d73 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.055659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c933ee4-890d-4228-a761-ce9caa9b13df None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.068459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98f34f-6f87-43c2-82e6-50b2397c3356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.069279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98f34f-6f87-43c2-82e6-50b2397c3356 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.069543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98f34f-6f87-43c2-82e6-50b2397c3356 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.069818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98f34f-6f87-43c2-82e6-50b2397c3356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.080893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98f34f-6f87-43c2-82e6-50b2397c3356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.082338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98f34f-6f87-43c2-82e6-50b2397c3356 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.092356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302b5a5-ee65-4b6c-8b72-4f470a0332a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.093059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302b5a5-ee65-4b6c-8b72-4f470a0332a1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.093339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302b5a5-ee65-4b6c-8b72-4f470a0332a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.093610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302b5a5-ee65-4b6c-8b72-4f470a0332a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.094327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302b5a5-ee65-4b6c-8b72-4f470a0332a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:34.094631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302b5a5-ee65-4b6c-8b72-4f470a0332a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.095980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302b5a5-ee65-4b6c-8b72-4f470a0332a1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-6f98f34f-6f87-43c2-82e6-50b2397c3356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:34.100729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:34.101595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:34.102469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:34.102469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:34.102469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:34.102469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:34.106796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 429/849] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:34.107692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98f34f-6f87-43c2-82e6-50b2397c3356 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.113205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 421/850] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:34.113205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302b5a5-ee65-4b6c-8b72-4f470a0332a1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.144098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:34.219175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98f34f-6f87-43c2-82e6-50b2397c3356 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.220491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:34.224138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302b5a5-ee65-4b6c-8b72-4f470a0332a1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.256200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba17ca1-3646-429a-9d19-769fee32d6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.257078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba17ca1-3646-429a-9d19-769fee32d6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.257366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba17ca1-3646-429a-9d19-769fee32d6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.257641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba17ca1-3646-429a-9d19-769fee32d6d3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.258319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba17ca1-3646-429a-9d19-769fee32d6d3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.259695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba17ca1-3646-429a-9d19-769fee32d6d3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.261220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24774007-c713-428b-890c-d96181566b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.261889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24774007-c713-428b-890c-d96181566b51 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.262174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24774007-c713-428b-890c-d96181566b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.262455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24774007-c713-428b-890c-d96181566b51 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.263188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24774007-c713-428b-890c-d96181566b51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:34.263498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24774007-c713-428b-890c-d96181566b51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.265549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24774007-c713-428b-890c-d96181566b51 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-fba17ca1-3646-429a-9d19-769fee32d6d3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:34.272707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:34.273885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:34.274800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 430/851] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:34.275652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba17ca1-3646-429a-9d19-769fee32d6d3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.282960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 422/852] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:34.283987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24774007-c713-428b-890c-d96181566b51 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.380056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:34.388259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba17ca1-3646-429a-9d19-769fee32d6d3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.399817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24774007-c713-428b-890c-d96181566b51 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.423494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda2db2-ce03-4c29-8b0e-e8b4ef6ce2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.424294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda2db2-ce03-4c29-8b0e-e8b4ef6ce2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.424572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda2db2-ce03-4c29-8b0e-e8b4ef6ce2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.424876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda2db2-ce03-4c29-8b0e-e8b4ef6ce2e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.432511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda2db2-ce03-4c29-8b0e-e8b4ef6ce2e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.433996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda2db2-ce03-4c29-8b0e-e8b4ef6ce2e3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.437166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e779f-d678-4f84-87ae-930e9fc505e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.437868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e779f-d678-4f84-87ae-930e9fc505e1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.438135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e779f-d678-4f84-87ae-930e9fc505e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.438416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e779f-d678-4f84-87ae-930e9fc505e1 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.439129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e779f-d678-4f84-87ae-930e9fc505e1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:34.439464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e779f-d678-4f84-87ae-930e9fc505e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.440846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e779f-d678-4f84-87ae-930e9fc505e1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.445470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 423/853] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:34.446246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e779f-d678-4f84-87ae-930e9fc505e1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7fda2db2-ce03-4c29-8b0e-e8b4ef6ce2e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:34.447816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:34.448696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:34.449676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 431/854] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:34.450605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb16dd-4def-45f7-a802-1d63677c5108 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.450889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb16dd-4def-45f7-a802-1d63677c5108 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.451167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb16dd-4def-45f7-a802-1d63677c5108 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.452814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 432/855] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:34.453378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb16dd-4def-45f7-a802-1d63677c5108 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.480558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:34.554353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb16dd-4def-45f7-a802-1d63677c5108 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.554544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e779f-d678-4f84-87ae-930e9fc505e1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.587639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ed5bd-d326-4035-bfbb-a736864036b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.588327 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ed5bd-d326-4035-bfbb-a736864036b4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.588611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ed5bd-d326-4035-bfbb-a736864036b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.588919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ed5bd-d326-4035-bfbb-a736864036b4 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.589630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ed5bd-d326-4035-bfbb-a736864036b4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:34.589882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0e316-90bb-43b3-90eb-c048c41e5008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.590015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ed5bd-d326-4035-bfbb-a736864036b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.590675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0e316-90bb-43b3-90eb-c048c41e5008 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.590955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0e316-90bb-43b3-90eb-c048c41e5008 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.591229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0e316-90bb-43b3-90eb-c048c41e5008 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.591322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ed5bd-d326-4035-bfbb-a736864036b4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.596520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 433/856] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:34.597673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ed5bd-d326-4035-bfbb-a736864036b4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.598024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0e316-90bb-43b3-90eb-c048c41e5008 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.599543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0e316-90bb-43b3-90eb-c048c41e5008 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-22c0e316-90bb-43b3-90eb-c048c41e5008 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:34.614299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:34.615269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:34.616843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:34.616843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:34.616843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:34.616843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:34.616843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:34.616843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:34.616843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:34.616843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:34.616843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 424/857] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:34.617364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5fb66-f36e-43fd-92ab-4905e9514c19 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.617551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5fb66-f36e-43fd-92ab-4905e9514c19 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.617889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5fb66-f36e-43fd-92ab-4905e9514c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.627323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:34.636671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ead5fb66-f36e-43fd-92ab-4905e9514c19 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:34.637919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ead5fb66-f36e-43fd-92ab-4905e9514c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:34.720139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ed5bd-d326-4035-bfbb-a736864036b4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.720999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 425/858] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:34.722180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5fb66-f36e-43fd-92ab-4905e9514c19 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.740384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:34.763949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae5dad-1b1d-484d-9a00-17e3c120a7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.764983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae5dad-1b1d-484d-9a00-17e3c120a7eb None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.765438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae5dad-1b1d-484d-9a00-17e3c120a7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.765869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae5dad-1b1d-484d-9a00-17e3c120a7eb None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.766885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae5dad-1b1d-484d-9a00-17e3c120a7eb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:34.767201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae5dad-1b1d-484d-9a00-17e3c120a7eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.768636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae5dad-1b1d-484d-9a00-17e3c120a7eb None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.789450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 434/859] 10.222.0.69 () {56 vars in 1135 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:34.790561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae5dad-1b1d-484d-9a00-17e3c120a7eb None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.836822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5fb66-f36e-43fd-92ab-4905e9514c19 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.872013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:34.873233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063b3f4-3254-4b3e-84d6-df033370c05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.873900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063b3f4-3254-4b3e-84d6-df033370c05d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.874185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063b3f4-3254-4b3e-84d6-df033370c05d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.874497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063b3f4-3254-4b3e-84d6-df033370c05d None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.875266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063b3f4-3254-4b3e-84d6-df033370c05d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:34.875614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063b3f4-3254-4b3e-84d6-df033370c05d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.877088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063b3f4-3254-4b3e-84d6-df033370c05d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.889623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 426/860] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:34.890483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063b3f4-3254-4b3e-84d6-df033370c05d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.903540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae5dad-1b1d-484d-9a00-17e3c120a7eb None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.938152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d1776-a7ae-44c4-bc03-5404469a1d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.939241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d1776-a7ae-44c4-bc03-5404469a1d60 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.939659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d1776-a7ae-44c4-bc03-5404469a1d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.940082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d1776-a7ae-44c4-bc03-5404469a1d60 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.940771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d1776-a7ae-44c4-bc03-5404469a1d60 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.942166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d1776-a7ae-44c4-bc03-5404469a1d60 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-673d1776-a7ae-44c4-bc03-5404469a1d60 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:34.954581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:34.956072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:34.957010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 435/861] 10.222.0.69 () {56 vars in 1132 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:34.957856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d1776-a7ae-44c4-bc03-5404469a1d60 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.981980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:35.008844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063b3f4-3254-4b3e-84d6-df033370c05d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:35.043168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b9b22-8e73-4df0-89af-a558d6088ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:35.043830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b9b22-8e73-4df0-89af-a558d6088ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:35.044111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b9b22-8e73-4df0-89af-a558d6088ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:35.044383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b9b22-8e73-4df0-89af-a558d6088ed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:35.066754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d1776-a7ae-44c4-bc03-5404469a1d60 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:35.103057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81679b-5205-4a69-8c9f-95d36ec7605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:35.103710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81679b-5205-4a69-8c9f-95d36ec7605e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:35.103989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81679b-5205-4a69-8c9f-95d36ec7605e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:35.104262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81679b-5205-4a69-8c9f-95d36ec7605e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:35.105135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81679b-5205-4a69-8c9f-95d36ec7605e None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=f2360d67304049278fe17c8abfe44a2b) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:35.105444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81679b-5205-4a69-8c9f-95d36ec7605e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:35.106864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81679b-5205-4a69-8c9f-95d36ec7605e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:35.112723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 436/862] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/role_assignments?user.id=f2360d67304049278fe17c8abfe44a2b&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:35.113510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81679b-5205-4a69-8c9f-95d36ec7605e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:35.117678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:35.131843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b9b22-8e73-4df0-89af-a558d6088ed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:35.134442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b9b22-8e73-4df0-89af-a558d6088ed9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:35.231155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:35.244286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81679b-5205-4a69-8c9f-95d36ec7605e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:35.269696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 427/863] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:23:35.275713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b9b22-8e73-4df0-89af-a558d6088ed9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:35.284524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25278b-0540-46fe-975c-999194710caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:35.285563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25278b-0540-46fe-975c-999194710caa None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:35.285563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25278b-0540-46fe-975c-999194710caa None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:35.285791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25278b-0540-46fe-975c-999194710caa None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:35.286452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25278b-0540-46fe-975c-999194710caa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:35.286754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25278b-0540-46fe-975c-999194710caa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:35.288234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25278b-0540-46fe-975c-999194710caa None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:35.293952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 437/864] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:35.299160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25278b-0540-46fe-975c-999194710caa None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:35.382717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:35.401787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b9b22-8e73-4df0-89af-a558d6088ed9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:35.427839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25278b-0540-46fe-975c-999194710caa None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:35.470883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c274e-3f97-4224-bcd5-b6194a2cc574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:35.471824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c274e-3f97-4224-bcd5-b6194a2cc574 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:35.472251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c274e-3f97-4224-bcd5-b6194a2cc574 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:35.472667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c274e-3f97-4224-bcd5-b6194a2cc574 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:35.473568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c274e-3f97-4224-bcd5-b6194a2cc574 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:35.474051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c274e-3f97-4224-bcd5-b6194a2cc574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:35.475598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c274e-3f97-4224-bcd5-b6194a2cc574 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:35.481412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 438/865] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:35.565466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cfcc1-b1da-4dee-af63-140c6bf70f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:35.566663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cfcc1-b1da-4dee-af63-140c6bf70f13 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:35.567110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cfcc1-b1da-4dee-af63-140c6bf70f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:35.567537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cfcc1-b1da-4dee-af63-140c6bf70f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:35.579729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cfcc1-b1da-4dee-af63-140c6bf70f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:35.581804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cfcc1-b1da-4dee-af63-140c6bf70f13 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1b5cfcc1-b1da-4dee-af63-140c6bf70f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:35.601143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:35.602306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:35.603402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:35.603402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:35.603402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:35.603402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:35.605380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 428/866] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:35.613933 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c274e-3f97-4224-bcd5-b6194a2cc574 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:35.719591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c274e-3f97-4224-bcd5-b6194a2cc574 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:35.750771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:35.757076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a104b-6f01-4d15-93f7-a4f0b148faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:35.758019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a104b-6f01-4d15-93f7-a4f0b148faf6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:35.758448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a104b-6f01-4d15-93f7-a4f0b148faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:35.758869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a104b-6f01-4d15-93f7-a4f0b148faf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:35.759783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a104b-6f01-4d15-93f7-a4f0b148faf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:35.760232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a104b-6f01-4d15-93f7-a4f0b148faf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:35.761829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a104b-6f01-4d15-93f7-a4f0b148faf6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:35.775966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 439/867] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:35.781383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cfcc1-b1da-4dee-af63-140c6bf70f13 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:35.814531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:35.887936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cfcc1-b1da-4dee-af63-140c6bf70f13 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:35.921898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04852966-c16c-48e9-858b-7e6f3ba12d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:35.922702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04852966-c16c-48e9-858b-7e6f3ba12d17 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:35.922985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04852966-c16c-48e9-858b-7e6f3ba12d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:35.923256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04852966-c16c-48e9-858b-7e6f3ba12d17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:35.930192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04852966-c16c-48e9-858b-7e6f3ba12d17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:35.931625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04852966-c16c-48e9-858b-7e6f3ba12d17 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-04852966-c16c-48e9-858b-7e6f3ba12d17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:35.945695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:35.946598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:35.947474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:35.947474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:35.947474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:35.947474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:35.947474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:35.947474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:35.947474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:35.947474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:35.947474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 429/868] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:35.953197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a104b-6f01-4d15-93f7-a4f0b148faf6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:36.063535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a104b-6f01-4d15-93f7-a4f0b148faf6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:36.098610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e86b65-b909-415b-9285-74f8002bee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:36.099270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e86b65-b909-415b-9285-74f8002bee04 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:36.099550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e86b65-b909-415b-9285-74f8002bee04 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:36.099828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e86b65-b909-415b-9285-74f8002bee04 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:36.100612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e86b65-b909-415b-9285-74f8002bee04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:36.100948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e86b65-b909-415b-9285-74f8002bee04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:36.102366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e86b65-b909-415b-9285-74f8002bee04 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:36.114877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 440/869] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:36.121827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04852966-c16c-48e9-858b-7e6f3ba12d17 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:36.212830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:36.228829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04852966-c16c-48e9-858b-7e6f3ba12d17 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:36.267463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fe8a8-4a81-4122-bca3-fd54f5875701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:36.268119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fe8a8-4a81-4122-bca3-fd54f5875701 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:36.268403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fe8a8-4a81-4122-bca3-fd54f5875701 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:36.268677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fe8a8-4a81-4122-bca3-fd54f5875701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:36.269544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fe8a8-4a81-4122-bca3-fd54f5875701 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=44d5bd3f9dfe45ecb8d1a51d248f3c54) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:36.269855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fe8a8-4a81-4122-bca3-fd54f5875701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:36.271270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fe8a8-4a81-4122-bca3-fd54f5875701 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:36.277836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 430/870] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:36 2022] GET /identity/v3/role_assignments?user.id=44d5bd3f9dfe45ecb8d1a51d248f3c54&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:36.675220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432126f-fcb0-400a-9d17-72017e990343 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:36.675603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432126f-fcb0-400a-9d17-72017e990343 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:36.675779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432126f-fcb0-400a-9d17-72017e990343 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:36.677437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 441/871] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:36.682666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e863c7-ad0c-4caf-a6ad-e0feb98c33b1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:36.683132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e863c7-ad0c-4caf-a6ad-e0feb98c33b1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:36.683548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e863c7-ad0c-4caf-a6ad-e0feb98c33b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:36.700558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-52e863c7-ad0c-4caf-a6ad-e0feb98c33b1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:36.702020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-52e863c7-ad0c-4caf-a6ad-e0feb98c33b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:36.772514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 431/872] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:36.795587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432126f-fcb0-400a-9d17-72017e990343 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:36.855096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:36.906308 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432126f-fcb0-400a-9d17-72017e990343 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:36.940481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a81eb7-ce83-4ec5-9e46-9354b5e6510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:36.941088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a81eb7-ce83-4ec5-9e46-9354b5e6510c None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:36.941369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a81eb7-ce83-4ec5-9e46-9354b5e6510c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:36.941721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a81eb7-ce83-4ec5-9e46-9354b5e6510c None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:36.942443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a81eb7-ce83-4ec5-9e46-9354b5e6510c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:36.943804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a81eb7-ce83-4ec5-9e46-9354b5e6510c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:36.961264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 442/873] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:23:36 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:37.411818 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5568a8-d2b4-4414-89a3-a262cdae11f5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:37.412358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5568a8-d2b4-4414-89a3-a262cdae11f5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:37.412358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5568a8-d2b4-4414-89a3-a262cdae11f5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:37.414075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 432/874] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:37.419849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8ee82-3a69-4f6c-b7d9-f0177b26a4ca None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:37.420515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8ee82-3a69-4f6c-b7d9-f0177b26a4ca None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:37.421046 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8ee82-3a69-4f6c-b7d9-f0177b26a4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:37.439416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1cb8ee82-3a69-4f6c-b7d9-f0177b26a4ca None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:37.440972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1cb8ee82-3a69-4f6c-b7d9-f0177b26a4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:37.517791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 443/875] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:37 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:37.542240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5568a8-d2b4-4414-89a3-a262cdae11f5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:37.550398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1147b-d1de-4563-b415-097ec1d38a03 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:37.550691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1147b-d1de-4563-b415-097ec1d38a03 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:37.550975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1147b-d1de-4563-b415-097ec1d38a03 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:37.552574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 444/876] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:37.557383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b5fe-0153-43a6-8017-975c253c1380 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:37.557675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b5fe-0153-43a6-8017-975c253c1380 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:37.557955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b5fe-0153-43a6-8017-975c253c1380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:37.574862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7b75b5fe-0153-43a6-8017-975c253c1380 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:37.576044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7b75b5fe-0153-43a6-8017-975c253c1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:37.607369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:37.652817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 445/877] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:37 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:37.665926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5568a8-d2b4-4414-89a3-a262cdae11f5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:37.670860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b5fe-0153-43a6-8017-975c253c1380 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:37.693161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:37.710165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8a4eb-6b37-4ba3-9cd7-0032cfcefae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:37.711071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8a4eb-6b37-4ba3-9cd7-0032cfcefae6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:37.711416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8a4eb-6b37-4ba3-9cd7-0032cfcefae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:37.711753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8a4eb-6b37-4ba3-9cd7-0032cfcefae6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:37.725743 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8a4eb-6b37-4ba3-9cd7-0032cfcefae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a5c8a4eb-6b37-4ba3-9cd7-0032cfcefae6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:37.736889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:37.738029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:23:37.739058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:37.741879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 433/878] 10.222.0.69 () {56 vars in 1120 bytes} [Fri Jun 24 10:23:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:37.748430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8a4eb-6b37-4ba3-9cd7-0032cfcefae6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:37.785341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b5fe-0153-43a6-8017-975c253c1380 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:37.822028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004bc93-6245-457a-8928-1a130bd0604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:37.824253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004bc93-6245-457a-8928-1a130bd0604a None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:37.824684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004bc93-6245-457a-8928-1a130bd0604a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:37.825128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004bc93-6245-457a-8928-1a130bd0604a None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:37.825997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004bc93-6245-457a-8928-1a130bd0604a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:37.827574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004bc93-6245-457a-8928-1a130bd0604a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:37.848198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 446/879] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:23:37 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:37.865281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8a4eb-6b37-4ba3-9cd7-0032cfcefae6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:37.911826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3c389-967d-4661-9a5c-d639b5920205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:37.912512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3c389-967d-4661-9a5c-d639b5920205 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:37.912821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3c389-967d-4661-9a5c-d639b5920205 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:37.913091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3c389-967d-4661-9a5c-d639b5920205 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:37.913837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3c389-967d-4661-9a5c-d639b5920205 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:37.914137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3c389-967d-4661-9a5c-d639b5920205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:37.915580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3c389-967d-4661-9a5c-d639b5920205 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:37.918565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:37.927115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 434/880] 10.222.0.69 () {56 vars in 1115 bytes} [Fri Jun 24 10:23:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:37.933215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004bc93-6245-457a-8928-1a130bd0604a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:37.966729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:38.054197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004bc93-6245-457a-8928-1a130bd0604a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:38.091537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb8604-b2f9-40d5-bea0-3fdb0ed31083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:38.092816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb8604-b2f9-40d5-bea0-3fdb0ed31083 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:38.093337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb8604-b2f9-40d5-bea0-3fdb0ed31083 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:38.093846 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb8604-b2f9-40d5-bea0-3fdb0ed31083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:38.106852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb8604-b2f9-40d5-bea0-3fdb0ed31083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:38.108960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb8604-b2f9-40d5-bea0-3fdb0ed31083 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-22fb8604-b2f9-40d5-bea0-3fdb0ed31083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:38.127320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:38.128268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:38.129166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:38.129166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:38.129166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:38.129166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:38.131477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 447/881] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:38.136718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3c389-967d-4661-9a5c-d639b5920205 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:38.240085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3c389-967d-4661-9a5c-d639b5920205 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:38.274315 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc64f-1c80-41cc-b3e4-cbfa3c651202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:38.274977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc64f-1c80-41cc-b3e4-cbfa3c651202 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:38.275256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc64f-1c80-41cc-b3e4-cbfa3c651202 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:38.275524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc64f-1c80-41cc-b3e4-cbfa3c651202 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:38.276248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bc64f-1c80-41cc-b3e4-cbfa3c651202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:38.276550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bc64f-1c80-41cc-b3e4-cbfa3c651202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:38.277937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bc64f-1c80-41cc-b3e4-cbfa3c651202 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:38.289859 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 435/882] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:38.296597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb8604-b2f9-40d5-bea0-3fdb0ed31083 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:38.385135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579963c-3896-48a9-b63d-491385f91df2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:38.385791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579963c-3896-48a9-b63d-491385f91df2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:38.386267 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579963c-3896-48a9-b63d-491385f91df2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:38.388061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 436/883] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:23:38.388164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:38.393525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1677a7-ae80-4e57-9dfb-2fa60fec8b3b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:38.394060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1677a7-ae80-4e57-9dfb-2fa60fec8b3b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:38.394552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1677a7-ae80-4e57-9dfb-2fa60fec8b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:38.411628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2a1677a7-ae80-4e57-9dfb-2fa60fec8b3b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:38.412843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2a1677a7-ae80-4e57-9dfb-2fa60fec8b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:38.413642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb8604-b2f9-40d5-bea0-3fdb0ed31083 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:38.451379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce7fb3-ae18-4aee-bf89-1ac442aacd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:38.452830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce7fb3-ae18-4aee-bf89-1ac442aacd91 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:38.454164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce7fb3-ae18-4aee-bf89-1ac442aacd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:38.454646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce7fb3-ae18-4aee-bf89-1ac442aacd91 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:38.468677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce7fb3-ae18-4aee-bf89-1ac442aacd91 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:38.471086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce7fb3-ae18-4aee-bf89-1ac442aacd91 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:38.477674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-e0ce7fb3-ae18-4aee-bf89-1ac442aacd91 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:38.491083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:38.492031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:38.492975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:38.493499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 437/884] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:38.495558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 448/885] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:23:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:38.504785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1677a7-ae80-4e57-9dfb-2fa60fec8b3b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:38.515942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce7fb3-ae18-4aee-bf89-1ac442aacd91 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:38.604425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1677a7-ae80-4e57-9dfb-2fa60fec8b3b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:38.629440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce7fb3-ae18-4aee-bf89-1ac442aacd91 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:38.641680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f62f5-4192-4d64-afb4-dde0bf70f819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:38.642443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f62f5-4192-4d64-afb4-dde0bf70f819 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:38.642730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f62f5-4192-4d64-afb4-dde0bf70f819 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:38.643007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f62f5-4192-4d64-afb4-dde0bf70f819 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:38.643863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f62f5-4192-4d64-afb4-dde0bf70f819 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:38.644172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f62f5-4192-4d64-afb4-dde0bf70f819 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:38.645761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f62f5-4192-4d64-afb4-dde0bf70f819 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:38.669547 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 438/886] 10.222.0.69 () {56 vars in 1149 bytes} [Fri Jun 24 10:23:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:38.670728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f62f5-4192-4d64-afb4-dde0bf70f819 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:38.683187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ce3c1-7f29-4f68-9153-8a4013b3da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:38.684765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ce3c1-7f29-4f68-9153-8a4013b3da3a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:38.685725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ce3c1-7f29-4f68-9153-8a4013b3da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:38.686429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ce3c1-7f29-4f68-9153-8a4013b3da3a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:38.687397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ce3c1-7f29-4f68-9153-8a4013b3da3a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:38.689731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ce3c1-7f29-4f68-9153-8a4013b3da3a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-9a4ce3c1-7f29-4f68-9153-8a4013b3da3a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:38.706951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:38.707914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:38.708769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:38.715729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 449/887] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:38.715965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ce3c1-7f29-4f68-9153-8a4013b3da3a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:38.720516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:38.730804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:38.787625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f62f5-4192-4d64-afb4-dde0bf70f819 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:38.823342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193fdc31-b0b8-44e0-8ee1-89eec75157d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:38.823999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fdc31-b0b8-44e0-8ee1-89eec75157d1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:38.824281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fdc31-b0b8-44e0-8ee1-89eec75157d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:38.824557 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fdc31-b0b8-44e0-8ee1-89eec75157d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:38.840606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ce3c1-7f29-4f68-9153-8a4013b3da3a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:38.876097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2710d-432f-4960-b174-2c34eecf4cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:38.877197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2710d-432f-4960-b174-2c34eecf4cda None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:38.877700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2710d-432f-4960-b174-2c34eecf4cda None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:38.878179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2710d-432f-4960-b174-2c34eecf4cda None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:38.879073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2710d-432f-4960-b174-2c34eecf4cda None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:38.879565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2710d-432f-4960-b174-2c34eecf4cda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:38.881189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2710d-432f-4960-b174-2c34eecf4cda None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:38.891220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 450/888] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:38.892460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2710d-432f-4960-b174-2c34eecf4cda None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:38.895630 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193fdc31-b0b8-44e0-8ee1-89eec75157d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:38.897418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193fdc31-b0b8-44e0-8ee1-89eec75157d1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:38.943684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:38.967926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:39.004006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 439/889] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:38 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:23:39.005131 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193fdc31-b0b8-44e0-8ee1-89eec75157d1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.011271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2710d-432f-4960-b174-2c34eecf4cda None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.054050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2645-559f-4504-9f10-c555467d1a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.055306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2645-559f-4504-9f10-c555467d1a8d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.055602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2645-559f-4504-9f10-c555467d1a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.055884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2645-559f-4504-9f10-c555467d1a8d None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.056867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2645-559f-4504-9f10-c555467d1a8d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:39.057182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2645-559f-4504-9f10-c555467d1a8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.059363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2645-559f-4504-9f10-c555467d1a8d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.065811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 451/890] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:39.070688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dee56-77a0-4def-8863-10dbd08c4189 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.070688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dee56-77a0-4def-8863-10dbd08c4189 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.070688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dee56-77a0-4def-8863-10dbd08c4189 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.070989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 452/891] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:39.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc57de4-647c-423b-ac2e-311ac81a6359 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.077798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc57de4-647c-423b-ac2e-311ac81a6359 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.078269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc57de4-647c-423b-ac2e-311ac81a6359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.097047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bcc57de4-647c-423b-ac2e-311ac81a6359 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:39.098570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bcc57de4-647c-423b-ac2e-311ac81a6359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:39.120801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193fdc31-b0b8-44e0-8ee1-89eec75157d1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.159249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76462ae-0739-45fd-83e7-fbc320036cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.160189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76462ae-0739-45fd-83e7-fbc320036cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.160486 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76462ae-0739-45fd-83e7-fbc320036cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.160902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76462ae-0739-45fd-83e7-fbc320036cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.172511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76462ae-0739-45fd-83e7-fbc320036cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.174160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76462ae-0739-45fd-83e7-fbc320036cd2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.182061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:39.182551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 453/892] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c76462ae-0739-45fd-83e7-fbc320036cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:39.197348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:39.198310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:39.199207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:39.199207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:39.199207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:39.199207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:39.199207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 440/893] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:39.203168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc57de4-647c-423b-ac2e-311ac81a6359 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.204661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76462ae-0739-45fd-83e7-fbc320036cd2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.276290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:39.318269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc57de4-647c-423b-ac2e-311ac81a6359 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.328478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76462ae-0739-45fd-83e7-fbc320036cd2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.355749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fe028-105c-42bf-b9fc-7f24490482f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.356561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fe028-105c-42bf-b9fc-7f24490482f3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.356869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fe028-105c-42bf-b9fc-7f24490482f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.357140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fe028-105c-42bf-b9fc-7f24490482f3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.358033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fe028-105c-42bf-b9fc-7f24490482f3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.359409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fe028-105c-42bf-b9fc-7f24490482f3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.370951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fbc60-da8e-40c1-8eee-9eb9a18ca44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.371622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fbc60-da8e-40c1-8eee-9eb9a18ca44d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-660fe028-105c-42bf-b9fc-7f24490482f3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:39.371747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:39.372651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:23:39.373769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:39.374506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fbc60-da8e-40c1-8eee-9eb9a18ca44d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.374506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fbc60-da8e-40c1-8eee-9eb9a18ca44d None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.374506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fbc60-da8e-40c1-8eee-9eb9a18ca44d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:39.374506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fbc60-da8e-40c1-8eee-9eb9a18ca44d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.374748 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fbc60-da8e-40c1-8eee-9eb9a18ca44d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.377919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 454/894] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:39.378912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fe028-105c-42bf-b9fc-7f24490482f3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.389077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 441/895] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:39.395133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fbc60-da8e-40c1-8eee-9eb9a18ca44d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.459915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:39.507019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fbc60-da8e-40c1-8eee-9eb9a18ca44d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.517134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fe028-105c-42bf-b9fc-7f24490482f3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.546391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41284328-1df7-481a-b621-61c660302f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.547824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41284328-1df7-481a-b621-61c660302f4d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.548316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41284328-1df7-481a-b621-61c660302f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.548846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41284328-1df7-481a-b621-61c660302f4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.550133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:39.558654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41284328-1df7-481a-b621-61c660302f4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.560044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e1c83-2463-4d33-8151-4baff9e66d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.560660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41284328-1df7-481a-b621-61c660302f4d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.560788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e1c83-2463-4d33-8151-4baff9e66d13 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.561083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e1c83-2463-4d33-8151-4baff9e66d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.561363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e1c83-2463-4d33-8151-4baff9e66d13 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.562277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e1c83-2463-4d33-8151-4baff9e66d13 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:39.562592 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e1c83-2463-4d33-8151-4baff9e66d13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.563972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e1c83-2463-4d33-8151-4baff9e66d13 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.570543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 455/896] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:39.575679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e1c83-2463-4d33-8151-4baff9e66d13 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-41284328-1df7-481a-b621-61c660302f4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:39.583126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:39.584111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:39.585009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:39.585009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:39.585009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:39.585009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:39.585009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:39.585009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:39.585009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:39.585009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:39.592076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 442/897] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:39.593190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41284328-1df7-481a-b621-61c660302f4d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.696074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41284328-1df7-481a-b621-61c660302f4d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.705974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e1c83-2463-4d33-8151-4baff9e66d13 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.715260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:39.735874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d28e4a-541c-433e-a6ec-d77c4f46d4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.736943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d28e4a-541c-433e-a6ec-d77c4f46d4ad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.737421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d28e4a-541c-433e-a6ec-d77c4f46d4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.737880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d28e4a-541c-433e-a6ec-d77c4f46d4ad None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.738857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d28e4a-541c-433e-a6ec-d77c4f46d4ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:39.739379 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d28e4a-541c-433e-a6ec-d77c4f46d4ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.741018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d28e4a-541c-433e-a6ec-d77c4f46d4ad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.750698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a6c22-00ac-4651-a654-e476b943f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.751708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a6c22-00ac-4651-a654-e476b943f885 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.752219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a6c22-00ac-4651-a654-e476b943f885 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.752711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a6c22-00ac-4651-a654-e476b943f885 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.753704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a6c22-00ac-4651-a654-e476b943f885 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:39.754378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a6c22-00ac-4651-a654-e476b943f885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.754507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 443/898] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:39.759873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d28e4a-541c-433e-a6ec-d77c4f46d4ad None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.760344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a6c22-00ac-4651-a654-e476b943f885 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.770858 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 456/899] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:39.772241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a6c22-00ac-4651-a654-e476b943f885 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.831224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:39.872020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d28e4a-541c-433e-a6ec-d77c4f46d4ad None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.877458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a6c22-00ac-4651-a654-e476b943f885 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:39.908497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d370583-c327-4ca4-8bca-4912fdc5093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.909169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370583-c327-4ca4-8bca-4912fdc5093c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.909455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370583-c327-4ca4-8bca-4912fdc5093c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.909728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370583-c327-4ca4-8bca-4912fdc5093c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.910558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d370583-c327-4ca4-8bca-4912fdc5093c None admin] RBAC: Adding query filter params (role.id=f6aab871c6aa416f9b085a37730a5202, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=f2360d67304049278fe17c8abfe44a2b) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:39.910866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d370583-c327-4ca4-8bca-4912fdc5093c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.912284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d370583-c327-4ca4-8bca-4912fdc5093c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.913943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4e605-381e-4e7b-b52b-b748fdc1401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:39.914839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4e605-381e-4e7b-b52b-b748fdc1401e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.915324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4e605-381e-4e7b-b52b-b748fdc1401e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.915792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4e605-381e-4e7b-b52b-b748fdc1401e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.916796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4e605-381e-4e7b-b52b-b748fdc1401e None admin] RBAC: Adding query filter params (interface=public, service_id=3fc356eaec41426697b45fd62679f193, region_id=RegionOne) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:39.917317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4e605-381e-4e7b-b52b-b748fdc1401e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:39.924773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 444/900] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/role_assignments?user.id=f2360d67304049278fe17c8abfe44a2b&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=f6aab871c6aa416f9b085a37730a5202 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:39.924974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4e605-381e-4e7b-b52b-b748fdc1401e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:39.930233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 457/901] 10.222.0.69 () {56 vars in 1251 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/endpoints?service_id=3fc356eaec41426697b45fd62679f193&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:39.934967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944809a5-94cb-4a90-86bf-f6f41f1d3b03 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.935254 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944809a5-94cb-4a90-86bf-f6f41f1d3b03 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.935529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944809a5-94cb-4a90-86bf-f6f41f1d3b03 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.936022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4e605-381e-4e7b-b52b-b748fdc1401e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:39.937230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 445/902] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:39.942794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35b61f-1c25-4125-ba5a-2b5697781f47 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:39.943108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35b61f-1c25-4125-ba5a-2b5697781f47 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:39.943342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35b61f-1c25-4125-ba5a-2b5697781f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:39.964358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5c35b61f-1c25-4125-ba5a-2b5697781f47 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:39.966156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5c35b61f-1c25-4125-ba5a-2b5697781f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:40.010774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:40.060789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 446/903] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:40.074657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4e605-381e-4e7b-b52b-b748fdc1401e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:40.083409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35b61f-1c25-4125-ba5a-2b5697781f47 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:40.103109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:40.129654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483c604-fc05-43f4-8358-1fc15b8b6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:40.130758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483c604-fc05-43f4-8358-1fc15b8b6b52 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:40.131072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483c604-fc05-43f4-8358-1fc15b8b6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:40.131356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483c604-fc05-43f4-8358-1fc15b8b6b52 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:40.132067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483c604-fc05-43f4-8358-1fc15b8b6b52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:40.134151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483c604-fc05-43f4-8358-1fc15b8b6b52 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:40.144891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 458/904] 10.222.0.69 () {56 vars in 1087 bytes} [Fri Jun 24 10:23:39 2022] GET /identity/v3/services => generated 1268 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:23:40.208270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35b61f-1c25-4125-ba5a-2b5697781f47 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:40.246202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876c99c-1dc4-4f4a-8a17-0469b0598723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:40.247044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876c99c-1dc4-4f4a-8a17-0469b0598723 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:40.247332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876c99c-1dc4-4f4a-8a17-0469b0598723 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:40.247608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876c99c-1dc4-4f4a-8a17-0469b0598723 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:40.248314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876c99c-1dc4-4f4a-8a17-0469b0598723 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:40.249848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876c99c-1dc4-4f4a-8a17-0469b0598723 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5876c99c-1dc4-4f4a-8a17-0469b0598723 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:40.263159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:40.264231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:40.265309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 447/905] 10.222.0.69 () {56 vars in 1132 bytes} [Fri Jun 24 10:23:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:40.272627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483c604-fc05-43f4-8358-1fc15b8b6b52 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:40.378677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483c604-fc05-43f4-8358-1fc15b8b6b52 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:40.414770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67f632-dca2-4bba-ac2c-e2a624408906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:40.415808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67f632-dca2-4bba-ac2c-e2a624408906 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:40.416311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67f632-dca2-4bba-ac2c-e2a624408906 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:40.416836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67f632-dca2-4bba-ac2c-e2a624408906 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:40.417750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67f632-dca2-4bba-ac2c-e2a624408906 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:40.418334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67f632-dca2-4bba-ac2c-e2a624408906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:40.420006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67f632-dca2-4bba-ac2c-e2a624408906 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:40.428764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 459/906] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:40.432010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876c99c-1dc4-4f4a-8a17-0469b0598723 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:40.485351 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:40.550221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876c99c-1dc4-4f4a-8a17-0469b0598723 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:40.585549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841faae-6140-43aa-904c-29479e2d7a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:40.586477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841faae-6140-43aa-904c-29479e2d7a35 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:40.586930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841faae-6140-43aa-904c-29479e2d7a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:40.587343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841faae-6140-43aa-904c-29479e2d7a35 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:40.588198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841faae-6140-43aa-904c-29479e2d7a35 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:40.588643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841faae-6140-43aa-904c-29479e2d7a35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:40.590187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841faae-6140-43aa-904c-29479e2d7a35 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:40.595835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 448/907] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:40.601630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67f632-dca2-4bba-ac2c-e2a624408906 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:40.650660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:40.711313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67f632-dca2-4bba-ac2c-e2a624408906 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:40.744891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308a8eb-0b75-45b7-870e-139c7a257aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:40.745547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308a8eb-0b75-45b7-870e-139c7a257aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:40.745831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308a8eb-0b75-45b7-870e-139c7a257aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:40.746102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308a8eb-0b75-45b7-870e-139c7a257aa3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:40.746892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308a8eb-0b75-45b7-870e-139c7a257aa3 None admin] RBAC: Adding query filter params (interface=public, service_id=fac5ff8617944968bfa7590536c67b00, region_id=RegionOne) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:40.747199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308a8eb-0b75-45b7-870e-139c7a257aa3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:40.748654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308a8eb-0b75-45b7-870e-139c7a257aa3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:40.754458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 460/908] 10.222.0.69 () {56 vars in 1251 bytes} [Fri Jun 24 10:23:40 2022] GET /identity/v3/endpoints?service_id=fac5ff8617944968bfa7590536c67b00&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:40.759823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841faae-6140-43aa-904c-29479e2d7a35 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:40.861441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:40.864993 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841faae-6140-43aa-904c-29479e2d7a35 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:40.905293 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a21c2-55a2-489f-81df-772528666b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:40.906369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a21c2-55a2-489f-81df-772528666b59 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:40.906834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a21c2-55a2-489f-81df-772528666b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:40.907289 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a21c2-55a2-489f-81df-772528666b59 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:40.908164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a21c2-55a2-489f-81df-772528666b59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:40.909754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a21c2-55a2-489f-81df-772528666b59 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:40.916181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 449/909] 10.222.0.69 () {56 vars in 1087 bytes} [Fri Jun 24 10:23:40 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:23:41.210819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09cb46-a2d1-47d3-be7d-5a6850cf9be3 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:41.211351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09cb46-a2d1-47d3-be7d-5a6850cf9be3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:41.211351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09cb46-a2d1-47d3-be7d-5a6850cf9be3 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:41.213045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 461/910] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:41.218475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902ac3f-9371-4910-b239-684ef70a51b2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:41.218766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902ac3f-9371-4910-b239-684ef70a51b2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:41.219047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902ac3f-9371-4910-b239-684ef70a51b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:41.236498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7902ac3f-9371-4910-b239-684ef70a51b2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:41.237719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7902ac3f-9371-4910-b239-684ef70a51b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:41.307273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 450/911] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:41.330394 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09cb46-a2d1-47d3-be7d-5a6850cf9be3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:41.441278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09cb46-a2d1-47d3-be7d-5a6850cf9be3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:41.454336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:41.484473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdf17d-8e0d-4f07-a258-c9bfab26d989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:41.485080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdf17d-8e0d-4f07-a258-c9bfab26d989 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:41.485359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdf17d-8e0d-4f07-a258-c9bfab26d989 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:41.485631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdf17d-8e0d-4f07-a258-c9bfab26d989 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:41.486361 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdf17d-8e0d-4f07-a258-c9bfab26d989 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:41.487716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdf17d-8e0d-4f07-a258-c9bfab26d989 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:41.502772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 462/912] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:23:41 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:42.026294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd38bb-0560-49b4-98e6-c66ecb0cb7a1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.026832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd38bb-0560-49b4-98e6-c66ecb0cb7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.026909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd38bb-0560-49b4-98e6-c66ecb0cb7a1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.028496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 451/913] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:42.034728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd70428-d081-45c9-be15-05a5c2c8c30a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.035314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd70428-d081-45c9-be15-05a5c2c8c30a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.035804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd70428-d081-45c9-be15-05a5c2c8c30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.053034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5dd70428-d081-45c9-be15-05a5c2c8c30a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:42.054565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5dd70428-d081-45c9-be15-05a5c2c8c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:42.132856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 463/914] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:42.151138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd38bb-0560-49b4-98e6-c66ecb0cb7a1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:42.254150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:42.262235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd38bb-0560-49b4-98e6-c66ecb0cb7a1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:42.296782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06727e06-e6a6-433c-b48d-5e5348100634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:42.297533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06727e06-e6a6-433c-b48d-5e5348100634 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.297817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06727e06-e6a6-433c-b48d-5e5348100634 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.298095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06727e06-e6a6-433c-b48d-5e5348100634 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.313172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06727e06-e6a6-433c-b48d-5e5348100634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-06727e06-e6a6-433c-b48d-5e5348100634 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:42.321352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:42.322277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:42.323100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 452/915] 10.222.0.69 () {56 vars in 1120 bytes} [Fri Jun 24 10:23:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:42.329135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd70428-d081-45c9-be15-05a5c2c8c30a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:42.424137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:42.430419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd70428-d081-45c9-be15-05a5c2c8c30a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:42.431384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305fb6f-f6c3-4b20-a14e-56d565f3691e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.431936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305fb6f-f6c3-4b20-a14e-56d565f3691e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.432409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305fb6f-f6c3-4b20-a14e-56d565f3691e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.434247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 453/916] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:42.445177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bcb53-a763-4004-a3ed-d08d723141d5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.445763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bcb53-a763-4004-a3ed-d08d723141d5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.446262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bcb53-a763-4004-a3ed-d08d723141d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.467796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237400e4-6be7-4beb-8a4d-3c736d9bc356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:42.468515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237400e4-6be7-4beb-8a4d-3c736d9bc356 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.468825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237400e4-6be7-4beb-8a4d-3c736d9bc356 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.469103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237400e4-6be7-4beb-8a4d-3c736d9bc356 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.469855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237400e4-6be7-4beb-8a4d-3c736d9bc356 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:42.470160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237400e4-6be7-4beb-8a4d-3c736d9bc356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:42.471533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237400e4-6be7-4beb-8a4d-3c736d9bc356 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:42.472073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-961bcb53-a763-4004-a3ed-d08d723141d5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:42.473699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-961bcb53-a763-4004-a3ed-d08d723141d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:42.480175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 464/917] 10.222.0.69 () {56 vars in 1115 bytes} [Fri Jun 24 10:23:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:42.490345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237400e4-6be7-4beb-8a4d-3c736d9bc356 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:42.558954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 454/918] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:42.582054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bcb53-a763-4004-a3ed-d08d723141d5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:42.610095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237400e4-6be7-4beb-8a4d-3c736d9bc356 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:42.648020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a071d-56d6-4cc5-a976-d5e99d5b5265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:42.649330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a071d-56d6-4cc5-a976-d5e99d5b5265 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.649813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a071d-56d6-4cc5-a976-d5e99d5b5265 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.650283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a071d-56d6-4cc5-a976-d5e99d5b5265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.662528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a071d-56d6-4cc5-a976-d5e99d5b5265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:42.664537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a071d-56d6-4cc5-a976-d5e99d5b5265 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-1e0a071d-56d6-4cc5-a976-d5e99d5b5265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:42.683526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:42.684454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:42.685819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:42.685819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:42.685819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:42.685819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:42.687864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 465/919] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:42.692638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:42.693512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a071d-56d6-4cc5-a976-d5e99d5b5265 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:42.709469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bcb53-a763-4004-a3ed-d08d723141d5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:42.711337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:42.747679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe2eea-435f-4662-b379-9a0b8bc6545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:42.749002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe2eea-435f-4662-b379-9a0b8bc6545b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.749583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe2eea-435f-4662-b379-9a0b8bc6545b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.750054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe2eea-435f-4662-b379-9a0b8bc6545b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.751000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe2eea-435f-4662-b379-9a0b8bc6545b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:42.752571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe2eea-435f-4662-b379-9a0b8bc6545b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d0fe2eea-435f-4662-b379-9a0b8bc6545b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:42.766555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:42.767578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:23:42.768439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:42.776355 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 455/920] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:23:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:42.777567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe2eea-435f-4662-b379-9a0b8bc6545b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:42.817444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a071d-56d6-4cc5-a976-d5e99d5b5265 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:42.859408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc8ac0-31a3-4d9f-bc9b-c2af3e2777b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:42.860088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc8ac0-31a3-4d9f-bc9b-c2af3e2777b2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.860380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc8ac0-31a3-4d9f-bc9b-c2af3e2777b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.860660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc8ac0-31a3-4d9f-bc9b-c2af3e2777b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.861772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc8ac0-31a3-4d9f-bc9b-c2af3e2777b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:42.862093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc8ac0-31a3-4d9f-bc9b-c2af3e2777b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:42.863795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc8ac0-31a3-4d9f-bc9b-c2af3e2777b2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:42.882996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 466/921] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:42.884766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc8ac0-31a3-4d9f-bc9b-c2af3e2777b2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:42.892158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe2eea-435f-4662-b379-9a0b8bc6545b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:42.934329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a5438-4556-4a8b-b9c3-77c07c1e3db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:42.935039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a5438-4556-4a8b-b9c3-77c07c1e3db2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:42.935321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a5438-4556-4a8b-b9c3-77c07c1e3db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:42.935638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a5438-4556-4a8b-b9c3-77c07c1e3db2 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:42.936472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a5438-4556-4a8b-b9c3-77c07c1e3db2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:42.936800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a5438-4556-4a8b-b9c3-77c07c1e3db2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:42.938663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a5438-4556-4a8b-b9c3-77c07c1e3db2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:42.949245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 456/922] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:23:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:42.951627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a5438-4556-4a8b-b9c3-77c07c1e3db2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:42.971716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:42.976939 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:43.009365 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc8ac0-31a3-4d9f-bc9b-c2af3e2777b2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.049404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457530ee-9676-4c1c-928c-bce9d31c4f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.050193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457530ee-9676-4c1c-928c-bce9d31c4f91 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.050481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457530ee-9676-4c1c-928c-bce9d31c4f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.050788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457530ee-9676-4c1c-928c-bce9d31c4f91 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.058544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457530ee-9676-4c1c-928c-bce9d31c4f91 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.060156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457530ee-9676-4c1c-928c-bce9d31c4f91 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.068301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a5438-4556-4a8b-b9c3-77c07c1e3db2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-457530ee-9676-4c1c-928c-bce9d31c4f91 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:43.075802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:43.077692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:43.078977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:43.082263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 467/923] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:23:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:43.083781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3be5c-f7fd-4c77-a3bf-c1809ad90caf None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.083922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3be5c-f7fd-4c77-a3bf-c1809ad90caf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.084138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3be5c-f7fd-4c77-a3bf-c1809ad90caf None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.085701 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 468/924] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:43.086410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3be5c-f7fd-4c77-a3bf-c1809ad90caf None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:43.111031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7102a-5b3a-419a-a5ec-078b7bcb1984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.111732 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7102a-5b3a-419a-a5ec-078b7bcb1984 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.111979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7102a-5b3a-419a-a5ec-078b7bcb1984 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.112282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7102a-5b3a-419a-a5ec-078b7bcb1984 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.113000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7102a-5b3a-419a-a5ec-078b7bcb1984 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:43.113325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7102a-5b3a-419a-a5ec-078b7bcb1984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.114676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7102a-5b3a-419a-a5ec-078b7bcb1984 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.124769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 457/925] 10.222.0.69 () {56 vars in 1114 bytes} [Fri Jun 24 10:23:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:43.124769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87ccd-f9bb-4296-b6f3-d1d773f98509 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.124769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87ccd-f9bb-4296-b6f3-d1d773f98509 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.124769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87ccd-f9bb-4296-b6f3-d1d773f98509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.137663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bab87ccd-f9bb-4296-b6f3-d1d773f98509 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:43.138910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bab87ccd-f9bb-4296-b6f3-d1d773f98509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:43.207693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3be5c-f7fd-4c77-a3bf-c1809ad90caf None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.220762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 458/926] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:43.220762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab87ccd-f9bb-4296-b6f3-d1d773f98509 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:43.227421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:43.241695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:43.248232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24e5e9-9588-4366-ad81-5f37b07b354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.249721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24e5e9-9588-4366-ad81-5f37b07b354a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.250144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24e5e9-9588-4366-ad81-5f37b07b354a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.250484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24e5e9-9588-4366-ad81-5f37b07b354a None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.251432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24e5e9-9588-4366-ad81-5f37b07b354a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:43.252091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24e5e9-9588-4366-ad81-5f37b07b354a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.253402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24e5e9-9588-4366-ad81-5f37b07b354a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.272290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 469/927] 10.222.0.69 () {56 vars in 1149 bytes} [Fri Jun 24 10:23:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:43.277640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24e5e9-9588-4366-ad81-5f37b07b354a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:43.327883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab87ccd-f9bb-4296-b6f3-d1d773f98509 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.364766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd13d-8931-4296-b338-24cdc34614fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.365447 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd13d-8931-4296-b338-24cdc34614fd None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.365732 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd13d-8931-4296-b338-24cdc34614fd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.366006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd13d-8931-4296-b338-24cdc34614fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.366747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdd13d-8931-4296-b338-24cdc34614fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.368128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdd13d-8931-4296-b338-24cdc34614fd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.400770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 459/928] 10.222.0.69 () {60 vars in 1144 bytes} [Fri Jun 24 10:23:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:43.400990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24e5e9-9588-4366-ad81-5f37b07b354a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.401419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd13d-8931-4296-b338-24cdc34614fd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:43.460961 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116f2cc-aeff-4156-bf2e-a6e2c8448723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.461979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116f2cc-aeff-4156-bf2e-a6e2c8448723 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.462357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116f2cc-aeff-4156-bf2e-a6e2c8448723 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.462727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116f2cc-aeff-4156-bf2e-a6e2c8448723 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.463591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116f2cc-aeff-4156-bf2e-a6e2c8448723 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.465429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116f2cc-aeff-4156-bf2e-a6e2c8448723 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.471468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-6116f2cc-aeff-4156-bf2e-a6e2c8448723 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:43.480133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:43.481252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:43.482349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 470/929] 10.222.0.69 () {56 vars in 1132 bytes} [Fri Jun 24 10:23:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:43.493873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116f2cc-aeff-4156-bf2e-a6e2c8448723 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:43.515845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd13d-8931-4296-b338-24cdc34614fd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.548605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:43.553442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9f3f4-7b5d-4101-b3ce-967c422e5354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.554255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9f3f4-7b5d-4101-b3ce-967c422e5354 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.554534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9f3f4-7b5d-4101-b3ce-967c422e5354 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.554807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9f3f4-7b5d-4101-b3ce-967c422e5354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.567214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9f3f4-7b5d-4101-b3ce-967c422e5354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.568687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9f3f4-7b5d-4101-b3ce-967c422e5354 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b3b9f3f4-7b5d-4101-b3ce-967c422e5354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:43.588961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:43.590066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:43.591203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:43.591203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:43.591203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:43.591203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:43.594845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 460/930] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:43.595744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9f3f4-7b5d-4101-b3ce-967c422e5354 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:43.611800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116f2cc-aeff-4156-bf2e-a6e2c8448723 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.650010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81159dc5-3ed1-44e3-bb0f-2f8c9ab20028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.650921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81159dc5-3ed1-44e3-bb0f-2f8c9ab20028 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.651342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81159dc5-3ed1-44e3-bb0f-2f8c9ab20028 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.651771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81159dc5-3ed1-44e3-bb0f-2f8c9ab20028 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.652484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81159dc5-3ed1-44e3-bb0f-2f8c9ab20028 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:43.652823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81159dc5-3ed1-44e3-bb0f-2f8c9ab20028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.654852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81159dc5-3ed1-44e3-bb0f-2f8c9ab20028 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.661130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 471/931] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:43.666983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81159dc5-3ed1-44e3-bb0f-2f8c9ab20028 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:43.707894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9f3f4-7b5d-4101-b3ce-967c422e5354 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.723778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:43.746411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11abd6-7390-46fd-aef1-37f893f36c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.747082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11abd6-7390-46fd-aef1-37f893f36c8b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.747367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11abd6-7390-46fd-aef1-37f893f36c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.747641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11abd6-7390-46fd-aef1-37f893f36c8b None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.748401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11abd6-7390-46fd-aef1-37f893f36c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:43.748708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11abd6-7390-46fd-aef1-37f893f36c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.750100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11abd6-7390-46fd-aef1-37f893f36c8b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.762440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 461/932] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:43.763553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ecb70-faca-46cb-9f7e-65a8363119bd None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.763896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ecb70-faca-46cb-9f7e-65a8363119bd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.764173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ecb70-faca-46cb-9f7e-65a8363119bd None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.765547 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 462/933] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:43.770202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ecb70-faca-46cb-9f7e-65a8363119bd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:43.780892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81159dc5-3ed1-44e3-bb0f-2f8c9ab20028 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.795216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:43.819407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e152d6-4445-4684-8e37-8037864342d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.820071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e152d6-4445-4684-8e37-8037864342d3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.820350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e152d6-4445-4684-8e37-8037864342d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.820621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e152d6-4445-4684-8e37-8037864342d3 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.821475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e152d6-4445-4684-8e37-8037864342d3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:43.821871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e152d6-4445-4684-8e37-8037864342d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.823229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e152d6-4445-4684-8e37-8037864342d3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.833521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 472/934] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:43.833521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5a984-f2a3-4acb-8bdb-46f92e6d6407 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.833521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5a984-f2a3-4acb-8bdb-46f92e6d6407 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.833521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5a984-f2a3-4acb-8bdb-46f92e6d6407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.848274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3ba5a984-f2a3-4acb-8bdb-46f92e6d6407 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:43.849457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3ba5a984-f2a3-4acb-8bdb-46f92e6d6407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:43.884949 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ecb70-faca-46cb-9f7e-65a8363119bd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:43.917823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 473/935] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:43.918725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5a984-f2a3-4acb-8bdb-46f92e6d6407 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:43.936632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d478fb2-ef20-489f-b011-8a8306fd3353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:43.937903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d478fb2-ef20-489f-b011-8a8306fd3353 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:43.938529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d478fb2-ef20-489f-b011-8a8306fd3353 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:43.938996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d478fb2-ef20-489f-b011-8a8306fd3353 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:43.949990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d478fb2-ef20-489f-b011-8a8306fd3353 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:43.951900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d478fb2-ef20-489f-b011-8a8306fd3353 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1d478fb2-ef20-489f-b011-8a8306fd3353 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:43.967981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:43.969190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 463/936] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:43.975100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d478fb2-ef20-489f-b011-8a8306fd3353 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:44.009995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:44.039710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5a984-f2a3-4acb-8bdb-46f92e6d6407 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:44.047579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:44.076267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047d5cd-a87f-426a-8e8b-639d7cd0cd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:44.076963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047d5cd-a87f-426a-8e8b-639d7cd0cd6c None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:44.077268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047d5cd-a87f-426a-8e8b-639d7cd0cd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:44.077525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047d5cd-a87f-426a-8e8b-639d7cd0cd6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:44.078353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047d5cd-a87f-426a-8e8b-639d7cd0cd6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:44.079675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047d5cd-a87f-426a-8e8b-639d7cd0cd6c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:44.081485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d478fb2-ef20-489f-b011-8a8306fd3353 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:44.102384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 474/937] 10.222.0.69 () {60 vars in 1144 bytes} [Fri Jun 24 10:23:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:44.103232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047d5cd-a87f-426a-8e8b-639d7cd0cd6c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:44.127003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab471e-3951-436d-a8d0-cf840cb296bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:44.128263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab471e-3951-436d-a8d0-cf840cb296bd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:44.128939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab471e-3951-436d-a8d0-cf840cb296bd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:44.129439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab471e-3951-436d-a8d0-cf840cb296bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:44.130445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab471e-3951-436d-a8d0-cf840cb296bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:44.132173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab471e-3951-436d-a8d0-cf840cb296bd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3bab471e-3951-436d-a8d0-cf840cb296bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:44.147320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:44.148357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:44.149226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:44.157441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 464/938] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:44.159189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab471e-3951-436d-a8d0-cf840cb296bd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:44.249276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047d5cd-a87f-426a-8e8b-639d7cd0cd6c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:44.281102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:44.291390 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab471e-3951-436d-a8d0-cf840cb296bd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:44.291668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8e2e-e0dd-49b0-9d8c-aa1f5b13f416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:44.292344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8e2e-e0dd-49b0-9d8c-aa1f5b13f416 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:44.292634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8e2e-e0dd-49b0-9d8c-aa1f5b13f416 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:44.293042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8e2e-e0dd-49b0-9d8c-aa1f5b13f416 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:44.293951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8e2e-e0dd-49b0-9d8c-aa1f5b13f416 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:44.294268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8e2e-e0dd-49b0-9d8c-aa1f5b13f416 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:44.296704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8e2e-e0dd-49b0-9d8c-aa1f5b13f416 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:44.310526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 475/939] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:44.316210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8e2e-e0dd-49b0-9d8c-aa1f5b13f416 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:44.334135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6291-8fff-4762-ac9c-1d522f2c4f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:44.334936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6291-8fff-4762-ac9c-1d522f2c4f29 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:44.335214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6291-8fff-4762-ac9c-1d522f2c4f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:44.335483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6291-8fff-4762-ac9c-1d522f2c4f29 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:44.336190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c6291-8fff-4762-ac9c-1d522f2c4f29 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:44.336493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c6291-8fff-4762-ac9c-1d522f2c4f29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:44.337872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c6291-8fff-4762-ac9c-1d522f2c4f29 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:44.346576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 465/940] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:44.347518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:44.354081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6291-8fff-4762-ac9c-1d522f2c4f29 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:44.433302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8e2e-e0dd-49b0-9d8c-aa1f5b13f416 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:44.460562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6291-8fff-4762-ac9c-1d522f2c4f29 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:44.479315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d50a7-c8a5-4e93-a45d-0e6097080560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:44.480298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d50a7-c8a5-4e93-a45d-0e6097080560 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:44.480642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d50a7-c8a5-4e93-a45d-0e6097080560 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:44.481152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d50a7-c8a5-4e93-a45d-0e6097080560 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/f2360d67304049278fe17c8abfe44a2b/roles/f6aab871c6aa416f9b085a37730a5202` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:44.485107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d50a7-c8a5-4e93-a45d-0e6097080560 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=f2360d67304049278fe17c8abfe44a2b, role_id=f6aab871c6aa416f9b085a37730a5202)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:44.486620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d50a7-c8a5-4e93-a45d-0e6097080560 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:44.498624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e278e-948c-4d90-859e-1420ac32ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:44.500209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e278e-948c-4d90-859e-1420ac32ef06 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:44.500317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e278e-948c-4d90-859e-1420ac32ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:44.500572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e278e-948c-4d90-859e-1420ac32ef06 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:44.501357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e278e-948c-4d90-859e-1420ac32ef06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:44.501677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e278e-948c-4d90-859e-1420ac32ef06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:44.503174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e278e-948c-4d90-859e-1420ac32ef06 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:44.506455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 476/941] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:23:44 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/f2360d67304049278fe17c8abfe44a2b/roles/f6aab871c6aa416f9b085a37730a5202 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:23:44.515737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 466/942] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:44.524490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d50a7-c8a5-4e93-a45d-0e6097080560 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:44.631280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:44.656484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d50a7-c8a5-4e93-a45d-0e6097080560 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:44.694254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6428d2-6e6f-43fd-9ed0-a88aad9820ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:44.694913 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6428d2-6e6f-43fd-9ed0-a88aad9820ff None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:44.695190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6428d2-6e6f-43fd-9ed0-a88aad9820ff None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:44.695461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6428d2-6e6f-43fd-9ed0-a88aad9820ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:44.696244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6428d2-6e6f-43fd-9ed0-a88aad9820ff None admin] RBAC: Adding query filter params (interface=public, service_id=5e658346ed804dcb8bc72cddde628aa0, region_id=RegionOne) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:44.696551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6428d2-6e6f-43fd-9ed0-a88aad9820ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:44.697948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6428d2-6e6f-43fd-9ed0-a88aad9820ff None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:44.703447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 477/943] 10.222.0.69 () {56 vars in 1251 bytes} [Fri Jun 24 10:23:44 2022] GET /identity/v3/endpoints?service_id=5e658346ed804dcb8bc72cddde628aa0&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:44.709188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e278e-948c-4d90-859e-1420ac32ef06 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:44.736264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:44.816195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e278e-948c-4d90-859e-1420ac32ef06 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:44.851652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85bb66-c0a8-4998-b076-0cd12035850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:44.852563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85bb66-c0a8-4998-b076-0cd12035850c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:44.853042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85bb66-c0a8-4998-b076-0cd12035850c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:44.853465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85bb66-c0a8-4998-b076-0cd12035850c None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:44.854296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85bb66-c0a8-4998-b076-0cd12035850c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:44.855820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85bb66-c0a8-4998-b076-0cd12035850c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:44.864498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 467/944] 10.222.0.69 () {56 vars in 1087 bytes} [Fri Jun 24 10:23:44 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:23:45.544527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26c2bc-61bd-4603-a9bf-052c414fa12c None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:45.545154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26c2bc-61bd-4603-a9bf-052c414fa12c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:45.545234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26c2bc-61bd-4603-a9bf-052c414fa12c None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:45.546814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 478/945] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:45.552068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765daa56-310a-4ea7-be40-3001c181bcf4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:45.552496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765daa56-310a-4ea7-be40-3001c181bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:45.552681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765daa56-310a-4ea7-be40-3001c181bcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:45.569791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-765daa56-310a-4ea7-be40-3001c181bcf4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:45.570957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-765daa56-310a-4ea7-be40-3001c181bcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:45.638150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 468/946] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:45.661216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26c2bc-61bd-4603-a9bf-052c414fa12c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:45.762998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26c2bc-61bd-4603-a9bf-052c414fa12c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:45.795953 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:45.796609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:45.796918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:45.797194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:45.797883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:45.799306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:45.800672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WARNING keystone.server.flask.common [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 10:23:45.818947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.sql.core [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 10:23:45.818947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 10:23:45.818947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [parameters: {'id': 'ce7ef05bc4344c348cae0d1ee8eb74e6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 10:23:45.818947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80145) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ce7ef05bc4344c348cae0d1ee8eb74e6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ce7ef05bc4344c348cae0d1ee8eb74e6. Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:23:45.826711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application result.read() Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 10:23:45.827569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application yield resource Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise self.value Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application session.commit() Jun 24 10:23:45.828497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.session.flush() Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application compat.raise_( Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 10:23:45.829424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 10:23:45.830314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application result.read() Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:23:45.831435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [parameters: {'id': 'ce7ef05bc4344c348cae0d1ee8eb74e6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:45.832278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:45.837259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ce7ef05bc4344c348cae0d1ee8eb74e6. Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:45.838195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 479/947] 10.222.0.69 () {60 vars in 1140 bytes} [Fri Jun 24 10:23:45 2022] POST /identity/v3/projects => generated 262 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:45.840597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765daa56-310a-4ea7-be40-3001c181bcf4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:45.860333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:45.947563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765daa56-310a-4ea7-be40-3001c181bcf4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:45.983760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c80e4-870f-493f-9c84-4639c3c30f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:45.984853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c80e4-870f-493f-9c84-4639c3c30f9a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:45.985332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c80e4-870f-493f-9c84-4639c3c30f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:45.985740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c80e4-870f-493f-9c84-4639c3c30f9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:45.993760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c80e4-870f-493f-9c84-4639c3c30f9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:45.995479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c80e4-870f-493f-9c84-4639c3c30f9a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-af9c80e4-870f-493f-9c84-4639c3c30f9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:46.011116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:46.012231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:46.013374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:46.013374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:46.013374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:46.013374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:46.013374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:46.013374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:46.013374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:46.013374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:46.015646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 469/948] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:46.021726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:46.029252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:46.128055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde3042-5f72-468f-a316-5fe394737099 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:46.162213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d366e-7640-4570-a561-af5906793624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:46.163240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d366e-7640-4570-a561-af5906793624 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.163693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d366e-7640-4570-a561-af5906793624 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.164203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d366e-7640-4570-a561-af5906793624 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.165208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d366e-7640-4570-a561-af5906793624 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:46.165684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d366e-7640-4570-a561-af5906793624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:46.167480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d366e-7640-4570-a561-af5906793624 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:46.188565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 480/949] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:46 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:46.267612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9efae-f667-45cf-afd8-64ade0055753 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.267897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9efae-f667-45cf-afd8-64ade0055753 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.268182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9efae-f667-45cf-afd8-64ade0055753 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.270030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 470/950] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:46.277998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0c4a3-05d5-4957-b9e3-c53b7ff99ce6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.277998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0c4a3-05d5-4957-b9e3-c53b7ff99ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.277998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0c4a3-05d5-4957-b9e3-c53b7ff99ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.295670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bde0c4a3-05d5-4957-b9e3-c53b7ff99ce6 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:46.297651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bde0c4a3-05d5-4957-b9e3-c53b7ff99ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:46.360724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:46.372125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 481/951] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:46.394567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9efae-f667-45cf-afd8-64ade0055753 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:46.485818 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:46.512372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9efae-f667-45cf-afd8-64ade0055753 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:46.547512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f563d5-1950-4551-8978-f5a96c1bee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:46.548239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f563d5-1950-4551-8978-f5a96c1bee9d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.548520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f563d5-1950-4551-8978-f5a96c1bee9d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.548830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f563d5-1950-4551-8978-f5a96c1bee9d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.549482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f563d5-1950-4551-8978-f5a96c1bee9d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:46.550847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f563d5-1950-4551-8978-f5a96c1bee9d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-15f563d5-1950-4551-8978-f5a96c1bee9d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:46.563520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:46.565074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:46.565966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 471/952] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:46.571119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0c4a3-05d5-4957-b9e3-c53b7ff99ce6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:46.601400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81092229-9ed4-43b9-91c1-1ddd103b06fc None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.601942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81092229-9ed4-43b9-91c1-1ddd103b06fc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.602223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81092229-9ed4-43b9-91c1-1ddd103b06fc None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.604260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 472/953] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:46.610438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860cd58-6a34-4002-9cb8-6289f5c75c8b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.611653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860cd58-6a34-4002-9cb8-6289f5c75c8b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.612044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860cd58-6a34-4002-9cb8-6289f5c75c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.633743 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b860cd58-6a34-4002-9cb8-6289f5c75c8b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:46.635550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b860cd58-6a34-4002-9cb8-6289f5c75c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:46.678500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0c4a3-05d5-4957-b9e3-c53b7ff99ce6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:46.708241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 473/954] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:46.718505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5c0fd-c7c6-4700-bfa1-0ab1af2e8681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:46.719174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5c0fd-c7c6-4700-bfa1-0ab1af2e8681 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.719459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5c0fd-c7c6-4700-bfa1-0ab1af2e8681 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.719734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5c0fd-c7c6-4700-bfa1-0ab1af2e8681 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.720446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5c0fd-c7c6-4700-bfa1-0ab1af2e8681 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:46.720777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5c0fd-c7c6-4700-bfa1-0ab1af2e8681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:46.722276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5c0fd-c7c6-4700-bfa1-0ab1af2e8681 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:46.727426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 482/955] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:46.733837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860cd58-6a34-4002-9cb8-6289f5c75c8b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:46.737454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5c0fd-c7c6-4700-bfa1-0ab1af2e8681 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:46.809215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:46.817454 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:46.849907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860cd58-6a34-4002-9cb8-6289f5c75c8b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:46.853983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5c0fd-c7c6-4700-bfa1-0ab1af2e8681 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:46.887484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0b5db-0a6e-4cf1-85b4-85c035bb8e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:46.888525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0b5db-0a6e-4cf1-85b4-85c035bb8e04 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.889087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0b5db-0a6e-4cf1-85b4-85c035bb8e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.889539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0b5db-0a6e-4cf1-85b4-85c035bb8e04 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.890427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0b5db-0a6e-4cf1-85b4-85c035bb8e04 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:46.890853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13da1d5-fb20-4cd5-bb2c-f816e2105b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:46.891136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0b5db-0a6e-4cf1-85b4-85c035bb8e04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:46.891789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13da1d5-fb20-4cd5-bb2c-f816e2105b0f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.892139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13da1d5-fb20-4cd5-bb2c-f816e2105b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.892485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13da1d5-fb20-4cd5-bb2c-f816e2105b0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.893282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0b5db-0a6e-4cf1-85b4-85c035bb8e04 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:46.899437 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 474/956] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:46.903804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13da1d5-fb20-4cd5-bb2c-f816e2105b0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f13da1d5-fb20-4cd5-bb2c-f816e2105b0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:46.912943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:46.914093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:46.915266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 483/957] 10.222.0.69 () {56 vars in 1120 bytes} [Fri Jun 24 10:23:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:46.920779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0b5db-0a6e-4cf1-85b4-85c035bb8e04 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:46.959112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaa7f0-4349-4da0-a3a8-43301adf364b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.959818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaa7f0-4349-4da0-a3a8-43301adf364b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.960428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaa7f0-4349-4da0-a3a8-43301adf364b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.962391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 484/958] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:46.970897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414e053-97e4-4843-b063-72d018b8ce39 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:46.971567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414e053-97e4-4843-b063-72d018b8ce39 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:46.972182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414e053-97e4-4843-b063-72d018b8ce39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:46.993296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b414e053-97e4-4843-b063-72d018b8ce39 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:46.994668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b414e053-97e4-4843-b063-72d018b8ce39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:47.047281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0b5db-0a6e-4cf1-85b4-85c035bb8e04 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:47.088850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 485/959] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:47.097634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8949-cc33-435a-8251-1a3ad65482da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:47.098608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8949-cc33-435a-8251-1a3ad65482da None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:47.099070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8949-cc33-435a-8251-1a3ad65482da None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:47.099541 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8949-cc33-435a-8251-1a3ad65482da None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:47.100479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff8949-cc33-435a-8251-1a3ad65482da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:47.100986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff8949-cc33-435a-8251-1a3ad65482da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:47.102527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff8949-cc33-435a-8251-1a3ad65482da None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:47.104247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:47.112983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 475/960] 10.222.0.69 () {56 vars in 1115 bytes} [Fri Jun 24 10:23:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:47.118667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414e053-97e4-4843-b063-72d018b8ce39 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:47.118904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8949-cc33-435a-8251-1a3ad65482da None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:47.167549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:47.226889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8949-cc33-435a-8251-1a3ad65482da None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:47.230268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414e053-97e4-4843-b063-72d018b8ce39 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:47.263107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980176a3-ea9e-4c01-a3e6-9c301a5ae4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:47.264426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980176a3-ea9e-4c01-a3e6-9c301a5ae4aa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:47.264915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980176a3-ea9e-4c01-a3e6-9c301a5ae4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:47.265423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980176a3-ea9e-4c01-a3e6-9c301a5ae4aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:47.266254 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980176a3-ea9e-4c01-a3e6-9c301a5ae4aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:47.268085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980176a3-ea9e-4c01-a3e6-9c301a5ae4aa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:47.270728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8f060-ee9a-4583-86a2-9738676ba9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:47.271984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8f060-ee9a-4583-86a2-9738676ba9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:47.272336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8f060-ee9a-4583-86a2-9738676ba9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:47.272697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8f060-ee9a-4583-86a2-9738676ba9b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-980176a3-ea9e-4c01-a3e6-9c301a5ae4aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:47.282542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:47.283700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:47.284533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:47.285307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8f060-ee9a-4583-86a2-9738676ba9b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:47.285924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8f060-ee9a-4583-86a2-9738676ba9b3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:47.292277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 476/961] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:47.293473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980176a3-ea9e-4c01-a3e6-9c301a5ae4aa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-e8e8f060-ee9a-4583-86a2-9738676ba9b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:47.306179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:47.307125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:47.307978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:47.307978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:47.307978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:47.307978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:47.312366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 486/962] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:47.313459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8f060-ee9a-4583-86a2-9738676ba9b3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:47.365330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:47.409762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980176a3-ea9e-4c01-a3e6-9c301a5ae4aa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:47.426180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8f060-ee9a-4583-86a2-9738676ba9b3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:47.431017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:47.455396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811b4aa-95b4-41f2-b844-8e3345151402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:47.456899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811b4aa-95b4-41f2-b844-8e3345151402 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:47.457421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811b4aa-95b4-41f2-b844-8e3345151402 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:47.457894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811b4aa-95b4-41f2-b844-8e3345151402 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:47.458904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811b4aa-95b4-41f2-b844-8e3345151402 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:47.459407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811b4aa-95b4-41f2-b844-8e3345151402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:47.461008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811b4aa-95b4-41f2-b844-8e3345151402 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:47.466992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 477/963] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:47.472439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811b4aa-95b4-41f2-b844-8e3345151402 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:47.485868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fc0fc-2d8c-4e9b-8b19-85a0e903f649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:47.486918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fc0fc-2d8c-4e9b-8b19-85a0e903f649 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:47.487504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fc0fc-2d8c-4e9b-8b19-85a0e903f649 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:47.487972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fc0fc-2d8c-4e9b-8b19-85a0e903f649 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:47.489045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fc0fc-2d8c-4e9b-8b19-85a0e903f649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:47.489566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fc0fc-2d8c-4e9b-8b19-85a0e903f649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:47.491141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fc0fc-2d8c-4e9b-8b19-85a0e903f649 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:47.508888 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 487/964] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:47.510111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fc0fc-2d8c-4e9b-8b19-85a0e903f649 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:47.580376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811b4aa-95b4-41f2-b844-8e3345151402 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:47.604560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:47.616071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca5b2-9bbe-4dea-ac43-80c02d95cbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:47.616668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca5b2-9bbe-4dea-ac43-80c02d95cbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:47.616987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca5b2-9bbe-4dea-ac43-80c02d95cbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:47.617261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca5b2-9bbe-4dea-ac43-80c02d95cbd5 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:47.617981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fca5b2-9bbe-4dea-ac43-80c02d95cbd5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:47.618290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fca5b2-9bbe-4dea-ac43-80c02d95cbd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:47.619660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fca5b2-9bbe-4dea-ac43-80c02d95cbd5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:47.629220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 478/965] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:47.629377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fc0fc-2d8c-4e9b-8b19-85a0e903f649 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:47.631420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca5b2-9bbe-4dea-ac43-80c02d95cbd5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:47.674097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f8b6c-0764-4571-9be9-61ba4c8499c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:47.674910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f8b6c-0764-4571-9be9-61ba4c8499c8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:47.675192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f8b6c-0764-4571-9be9-61ba4c8499c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:47.676243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f8b6c-0764-4571-9be9-61ba4c8499c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:47.687578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f8b6c-0764-4571-9be9-61ba4c8499c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:47.689409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f8b6c-0764-4571-9be9-61ba4c8499c8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-6c6f8b6c-0764-4571-9be9-61ba4c8499c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:47.710967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:47.711963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:23:47.712955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:47.719083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 488/966] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:23:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:47.719924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f8b6c-0764-4571-9be9-61ba4c8499c8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:47.743739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:47.767734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca5b2-9bbe-4dea-ac43-80c02d95cbd5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:47.802121 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d814d2-3050-44d0-a39b-95059abc6391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:47.802773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d814d2-3050-44d0-a39b-95059abc6391 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:47.803052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d814d2-3050-44d0-a39b-95059abc6391 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:47.803326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d814d2-3050-44d0-a39b-95059abc6391 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:47.804062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d814d2-3050-44d0-a39b-95059abc6391 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:47.805487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d814d2-3050-44d0-a39b-95059abc6391 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:47.833268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 479/967] 10.222.0.69 () {60 vars in 1144 bytes} [Fri Jun 24 10:23:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:47.849613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f8b6c-0764-4571-9be9-61ba4c8499c8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:47.896138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c539d7-a1a8-4ebd-9a5e-31a0319ccd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:47.897224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c539d7-a1a8-4ebd-9a5e-31a0319ccd7b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:47.897726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c539d7-a1a8-4ebd-9a5e-31a0319ccd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:47.898207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c539d7-a1a8-4ebd-9a5e-31a0319ccd7b None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:47.899263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c539d7-a1a8-4ebd-9a5e-31a0319ccd7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:47.899781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c539d7-a1a8-4ebd-9a5e-31a0319ccd7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:47.901498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c539d7-a1a8-4ebd-9a5e-31a0319ccd7b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:47.923363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 489/968] 10.222.0.69 () {56 vars in 1149 bytes} [Fri Jun 24 10:23:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:47.928898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d814d2-3050-44d0-a39b-95059abc6391 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:47.985009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:48.043066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d814d2-3050-44d0-a39b-95059abc6391 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:48.083855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e20198-9959-4e9a-a106-f1075a1d72f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:48.084517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e20198-9959-4e9a-a106-f1075a1d72f5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:48.084824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e20198-9959-4e9a-a106-f1075a1d72f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:48.085097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e20198-9959-4e9a-a106-f1075a1d72f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:48.156336 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e20198-9959-4e9a-a106-f1075a1d72f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:48.157859 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e20198-9959-4e9a-a106-f1075a1d72f5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:48.203866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:48.261133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 480/969] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:47 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:23:48.266490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c539d7-a1a8-4ebd-9a5e-31a0319ccd7b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:48.343792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:48.374662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c539d7-a1a8-4ebd-9a5e-31a0319ccd7b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:48.408980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b80e4-8a03-4dd1-9d25-2746d5be8794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:48.409704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b80e4-8a03-4dd1-9d25-2746d5be8794 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:48.409980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b80e4-8a03-4dd1-9d25-2746d5be8794 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:48.410254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b80e4-8a03-4dd1-9d25-2746d5be8794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:48.410930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259880f-8297-4d50-94d5-323f0be5ba6c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:48.411220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259880f-8297-4d50-94d5-323f0be5ba6c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:48.411492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259880f-8297-4d50-94d5-323f0be5ba6c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:48.414134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 481/970] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:23:48.418242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41987ee9-9f31-491e-8534-5f65768b8059 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:48.418522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41987ee9-9f31-491e-8534-5f65768b8059 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:48.418797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41987ee9-9f31-491e-8534-5f65768b8059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:48.421528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b80e4-8a03-4dd1-9d25-2746d5be8794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:48.422992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b80e4-8a03-4dd1-9d25-2746d5be8794 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-198b80e4-8a03-4dd1-9d25-2746d5be8794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:48.442530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:48.443489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:48.444381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:48.444381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:48.444381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:48.444381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:48.444381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 490/971] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:48.444655 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-41987ee9-9f31-491e-8534-5f65768b8059 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:48.445693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-41987ee9-9f31-491e-8534-5f65768b8059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:48.449797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b80e4-8a03-4dd1-9d25-2746d5be8794 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:48.524842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 482/972] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:48.544427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41987ee9-9f31-491e-8534-5f65768b8059 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:48.560851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b80e4-8a03-4dd1-9d25-2746d5be8794 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:48.584568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:48.595647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a4cf-0a38-42b5-b254-159876101c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:48.596691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a4cf-0a38-42b5-b254-159876101c05 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:48.597061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a4cf-0a38-42b5-b254-159876101c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:48.597424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a4cf-0a38-42b5-b254-159876101c05 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:48.598223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63a4cf-0a38-42b5-b254-159876101c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:48.598589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63a4cf-0a38-42b5-b254-159876101c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:48.600019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63a4cf-0a38-42b5-b254-159876101c05 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:48.617082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 491/973] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:48.618044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a4cf-0a38-42b5-b254-159876101c05 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:48.627674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:48.656464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41987ee9-9f31-491e-8534-5f65768b8059 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:48.694941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:48.695572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:48.695853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:48.696165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:48.696915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:48.698505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:48.699935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING keystone.server.flask.common [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 10:23:48.714375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.sql.core [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 10:23:48.714375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 10:23:48.714375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [parameters: {'user_id': '12f0cf5e4b5c4d73b84618f684b5e61d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 10:23:48.714375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 10:23:48.722400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a4cf-0a38-42b5-b254-159876101c05 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result.read() Jun 24 10:23:48.737467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:23:48.739322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application yield resource Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise self.value Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application session.commit() Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 10:23:48.740586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.session.flush() Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application compat.raise_( Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 10:23:48.742016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:23:48.743117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result.read() Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:23:48.744143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [parameters: {'user_id': '12f0cf5e4b5c4d73b84618f684b5e61d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:48.745301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 10:23:48.746263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 10:23:48.747358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 10:23:48.747358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 10:23:48.747358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 10:23:48.747358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 10:23:48.747358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 10:23:48.747358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:48.747358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 483/974] 10.222.0.69 () {60 vars in 1131 bytes} [Fri Jun 24 10:23:48 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:48.755116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:48.778280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b80c0-6cff-469b-ab53-0e7fa22e2756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:48.779700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b80c0-6cff-469b-ab53-0e7fa22e2756 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:48.780057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b80c0-6cff-469b-ab53-0e7fa22e2756 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:48.780455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b80c0-6cff-469b-ab53-0e7fa22e2756 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:48.790580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b80c0-6cff-469b-ab53-0e7fa22e2756 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:48.792598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b80c0-6cff-469b-ab53-0e7fa22e2756 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-242b80c0-6cff-469b-ab53-0e7fa22e2756 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:48.813200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:48.814116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:48.815315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:48.815315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:48.815315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:48.815315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:48.815315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:48.815315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:48.815315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:48.815315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:48.820089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 492/975] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:48.821048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b80c0-6cff-469b-ab53-0e7fa22e2756 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:48.880505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b7833-db8d-460c-85f2-440d3fceb08e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:48.882887 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:48.928290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131b24f-69fa-455f-b678-e5875316387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:48.929768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131b24f-69fa-455f-b678-e5875316387d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:48.930058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131b24f-69fa-455f-b678-e5875316387d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:48.930335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131b24f-69fa-455f-b678-e5875316387d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:48.934089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:48.942661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131b24f-69fa-455f-b678-e5875316387d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:48.945651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131b24f-69fa-455f-b678-e5875316387d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:48.947872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b80c0-6cff-469b-ab53-0e7fa22e2756 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a131b24f-69fa-455f-b678-e5875316387d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:48.965384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:48.966556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:23:48.967811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:48.970613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 484/976] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:23:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:48.976760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131b24f-69fa-455f-b678-e5875316387d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:48.987857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f3917-9055-4ddb-88fb-f5e1f3fb129f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:48.988524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f3917-9055-4ddb-88fb-f5e1f3fb129f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:48.988839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f3917-9055-4ddb-88fb-f5e1f3fb129f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:48.989154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f3917-9055-4ddb-88fb-f5e1f3fb129f None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:48.990075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f3917-9055-4ddb-88fb-f5e1f3fb129f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:48.990389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f3917-9055-4ddb-88fb-f5e1f3fb129f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:48.992171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f3917-9055-4ddb-88fb-f5e1f3fb129f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:49.005191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 493/977] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:49.010414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c1970-010e-4f2d-b0d5-b4977cedceb2 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:49.010715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c1970-010e-4f2d-b0d5-b4977cedceb2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:49.011040 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c1970-010e-4f2d-b0d5-b4977cedceb2 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:49.012323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 494/978] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:49.013036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c1970-010e-4f2d-b0d5-b4977cedceb2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:49.089091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131b24f-69fa-455f-b678-e5875316387d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:49.127326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c1970-010e-4f2d-b0d5-b4977cedceb2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:49.128340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62375883-1b25-4e19-b30c-afac5e8ecd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:49.129062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62375883-1b25-4e19-b30c-afac5e8ecd43 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:49.129342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62375883-1b25-4e19-b30c-afac5e8ecd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:49.130356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62375883-1b25-4e19-b30c-afac5e8ecd43 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:49.130466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62375883-1b25-4e19-b30c-afac5e8ecd43 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:49.130715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62375883-1b25-4e19-b30c-afac5e8ecd43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:49.131067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:49.132130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62375883-1b25-4e19-b30c-afac5e8ecd43 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:49.165040 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af83d72-4d3f-4dd4-860f-79af137b7d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:49.165689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af83d72-4d3f-4dd4-860f-79af137b7d93 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:49.165973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af83d72-4d3f-4dd4-860f-79af137b7d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:49.166251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af83d72-4d3f-4dd4-860f-79af137b7d93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:49.167208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af83d72-4d3f-4dd4-860f-79af137b7d93 None admin] RBAC: Adding query filter params (role.id=f6aab871c6aa416f9b085a37730a5202, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=f2360d67304049278fe17c8abfe44a2b) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:49.167521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af83d72-4d3f-4dd4-860f-79af137b7d93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:49.168970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af83d72-4d3f-4dd4-860f-79af137b7d93 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:49.177297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 485/979] 10.222.0.69 () {56 vars in 1101 bytes} [Fri Jun 24 10:23:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:49.177297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8790c72-4ed1-460f-8632-9971d2e2fa98 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:49.177297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8790c72-4ed1-460f-8632-9971d2e2fa98 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:49.177297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8790c72-4ed1-460f-8632-9971d2e2fa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:49.183606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 495/980] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:49 2022] GET /identity/v3/role_assignments?user.id=f2360d67304049278fe17c8abfe44a2b&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=f6aab871c6aa416f9b085a37730a5202 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:49.195835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d8790c72-4ed1-460f-8632-9971d2e2fa98 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:49.197653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d8790c72-4ed1-460f-8632-9971d2e2fa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:49.257083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:49.279091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 486/981] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:49.303116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af83d72-4d3f-4dd4-860f-79af137b7d93 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:49.413033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af83d72-4d3f-4dd4-860f-79af137b7d93 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:49.454323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdcde37-287a-408a-8e2f-30425c98e7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:49.455331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcde37-287a-408a-8e2f-30425c98e7ba None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:49.456156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcde37-287a-408a-8e2f-30425c98e7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:49.456620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcde37-287a-408a-8e2f-30425c98e7ba None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:49.457599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcde37-287a-408a-8e2f-30425c98e7ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:49.459138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcde37-287a-408a-8e2f-30425c98e7ba None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:49.477644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 496/982] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:23:49 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:50.043508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dec0d-2f0e-4744-978b-d1eddd6b0603 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:50.044078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dec0d-2f0e-4744-978b-d1eddd6b0603 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:50.044078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dec0d-2f0e-4744-978b-d1eddd6b0603 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:50.045798 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 487/983] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:50.051063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1a217-a4ca-46b7-845a-ce1faaf47262 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:50.051471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1a217-a4ca-46b7-845a-ce1faaf47262 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:50.051650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1a217-a4ca-46b7-845a-ce1faaf47262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:50.069382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8fa1a217-a4ca-46b7-845a-ce1faaf47262 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:50.070587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8fa1a217-a4ca-46b7-845a-ce1faaf47262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:50.096182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:50.150693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 497/984] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:50.173595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dec0d-2f0e-4744-978b-d1eddd6b0603 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:50.280286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dec0d-2f0e-4744-978b-d1eddd6b0603 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:50.315104 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b2e7-da17-4bd7-a108-4f0ffc18776c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:50.315835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b2e7-da17-4bd7-a108-4f0ffc18776c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:50.316118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b2e7-da17-4bd7-a108-4f0ffc18776c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:50.316391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b2e7-da17-4bd7-a108-4f0ffc18776c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:50.317075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0b2e7-da17-4bd7-a108-4f0ffc18776c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:50.318444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0b2e7-da17-4bd7-a108-4f0ffc18776c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6dd0b2e7-da17-4bd7-a108-4f0ffc18776c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:50.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:50.331813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:50.332628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:50.336863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 488/985] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:50.338143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1a217-a4ca-46b7-845a-ce1faaf47262 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:50.445677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1a217-a4ca-46b7-845a-ce1faaf47262 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:50.485343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93872b04-204d-4197-a640-24b5acb3c285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:50.485969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93872b04-204d-4197-a640-24b5acb3c285 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:50.486255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93872b04-204d-4197-a640-24b5acb3c285 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:50.486535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93872b04-204d-4197-a640-24b5acb3c285 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:50.487249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93872b04-204d-4197-a640-24b5acb3c285 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:50.487560 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93872b04-204d-4197-a640-24b5acb3c285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:50.488966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93872b04-204d-4197-a640-24b5acb3c285 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:50.494103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 498/986] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:50.499904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b2e7-da17-4bd7-a108-4f0ffc18776c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:50.550781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:50.605621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b2e7-da17-4bd7-a108-4f0ffc18776c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:50.639375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac0634-19a5-4bf5-bca2-72ca36bb5f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:50.640043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac0634-19a5-4bf5-bca2-72ca36bb5f7f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:50.640327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac0634-19a5-4bf5-bca2-72ca36bb5f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:50.640605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac0634-19a5-4bf5-bca2-72ca36bb5f7f None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:50.641350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac0634-19a5-4bf5-bca2-72ca36bb5f7f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:50.641656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac0634-19a5-4bf5-bca2-72ca36bb5f7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:50.643034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac0634-19a5-4bf5-bca2-72ca36bb5f7f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:50.648366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 489/987] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:50.654103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93872b04-204d-4197-a640-24b5acb3c285 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:50.673665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:50.756093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93872b04-204d-4197-a640-24b5acb3c285 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:50.790388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37fa62-32b1-4c52-bc71-71861859c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:50.791049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37fa62-32b1-4c52-bc71-71861859c558 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:50.791334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37fa62-32b1-4c52-bc71-71861859c558 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:50.791613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37fa62-32b1-4c52-bc71-71861859c558 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:50.792417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37fa62-32b1-4c52-bc71-71861859c558 None admin] RBAC: Adding query filter params (interface=admin, service_id=5e658346ed804dcb8bc72cddde628aa0, region_id=RegionOne) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:50.792762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37fa62-32b1-4c52-bc71-71861859c558 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:50.794138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37fa62-32b1-4c52-bc71-71861859c558 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:50.799975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 499/988] 10.222.0.69 () {56 vars in 1249 bytes} [Fri Jun 24 10:23:50 2022] GET /identity/v3/endpoints?service_id=5e658346ed804dcb8bc72cddde628aa0&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:50.805341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac0634-19a5-4bf5-bca2-72ca36bb5f7f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:50.907440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac0634-19a5-4bf5-bca2-72ca36bb5f7f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:50.920556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:50.944726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94f33b-cf47-4b96-851f-2bd8c2ca176f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:50.945707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94f33b-cf47-4b96-851f-2bd8c2ca176f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:50.946139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94f33b-cf47-4b96-851f-2bd8c2ca176f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:50.946560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94f33b-cf47-4b96-851f-2bd8c2ca176f None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:50.947501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94f33b-cf47-4b96-851f-2bd8c2ca176f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:50.949202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94f33b-cf47-4b96-851f-2bd8c2ca176f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:50.956786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 490/989] 10.222.0.69 () {56 vars in 1087 bytes} [Fri Jun 24 10:23:50 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:23:51.258047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ba07-30e9-455d-a04e-cfe4e897ad60 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:51.258572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ba07-30e9-455d-a04e-cfe4e897ad60 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:51.258572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ba07-30e9-455d-a04e-cfe4e897ad60 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:51.260037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 500/990] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:51.265455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387d8a7-b8ea-4a97-948c-27f172b301a5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:51.265887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387d8a7-b8ea-4a97-948c-27f172b301a5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:51.265960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387d8a7-b8ea-4a97-948c-27f172b301a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:51.283067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e387d8a7-b8ea-4a97-948c-27f172b301a5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:51.284246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e387d8a7-b8ea-4a97-948c-27f172b301a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:51.353863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 491/991] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:51.376388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ba07-30e9-455d-a04e-cfe4e897ad60 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:51.473844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:51.489002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0056dd2-e872-4809-9f44-f163481ae5f3 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:51.489288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0056dd2-e872-4809-9f44-f163481ae5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:51.489552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0056dd2-e872-4809-9f44-f163481ae5f3 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:51.491205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 492/992] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:51.495956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ba07-30e9-455d-a04e-cfe4e897ad60 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:51.496126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af99f6a-d5bc-4b7b-bfe6-0c4b562e5057 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:51.496504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af99f6a-d5bc-4b7b-bfe6-0c4b562e5057 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:51.496875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af99f6a-d5bc-4b7b-bfe6-0c4b562e5057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:51.516042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4af99f6a-d5bc-4b7b-bfe6-0c4b562e5057 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:51.517295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4af99f6a-d5bc-4b7b-bfe6-0c4b562e5057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:51.531692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac2699-54b6-4819-b8cc-e196e417e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:51.532597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac2699-54b6-4819-b8cc-e196e417e09b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:51.532905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac2699-54b6-4819-b8cc-e196e417e09b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:51.533179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac2699-54b6-4819-b8cc-e196e417e09b None admin] PATH_INFO: `/v3/services/image` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:51.533833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac2699-54b6-4819-b8cc-e196e417e09b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:51.535308 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac2699-54b6-4819-b8cc-e196e417e09b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d0ac2699-54b6-4819-b8cc-e196e417e09b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:51.547938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:51.549112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:51.550444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 501/993] 10.222.0.69 () {56 vars in 1105 bytes} [Fri Jun 24 10:23:51 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:51.554836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac2699-54b6-4819-b8cc-e196e417e09b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:51.589697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 493/994] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:51.612857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af99f6a-d5bc-4b7b-bfe6-0c4b562e5057 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:51.647402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:51.663210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac2699-54b6-4819-b8cc-e196e417e09b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:51.698672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c546d4-03f1-49a3-8cab-5ce24a972f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:51.699716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c546d4-03f1-49a3-8cab-5ce24a972f2e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:51.700190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c546d4-03f1-49a3-8cab-5ce24a972f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:51.700684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c546d4-03f1-49a3-8cab-5ce24a972f2e None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:51.701717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c546d4-03f1-49a3-8cab-5ce24a972f2e None admin] RBAC: Adding query filter params (name=image) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:51.702305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c546d4-03f1-49a3-8cab-5ce24a972f2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:51.703873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c546d4-03f1-49a3-8cab-5ce24a972f2e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:51.709474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 502/995] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:51.714864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c546d4-03f1-49a3-8cab-5ce24a972f2e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:51.722555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af99f6a-d5bc-4b7b-bfe6-0c4b562e5057 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:51.727941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:51.758390 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd1fe7-9cf9-4b56-aedc-0f192ea8ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:51.759188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd1fe7-9cf9-4b56-aedc-0f192ea8ea11 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:51.759469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd1fe7-9cf9-4b56-aedc-0f192ea8ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:51.759738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd1fe7-9cf9-4b56-aedc-0f192ea8ea11 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:51.760392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd1fe7-9cf9-4b56-aedc-0f192ea8ea11 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:51.761779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd1fe7-9cf9-4b56-aedc-0f192ea8ea11 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-defd1fe7-9cf9-4b56-aedc-0f192ea8ea11 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:51.773964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:51.774933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:51.775791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 494/996] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:51.780940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd1fe7-9cf9-4b56-aedc-0f192ea8ea11 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:51.823046 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c546d4-03f1-49a3-8cab-5ce24a972f2e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:51.858148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cde4e-68b8-4773-9cec-8ef78203b063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:51.858812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cde4e-68b8-4773-9cec-8ef78203b063 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:51.859099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cde4e-68b8-4773-9cec-8ef78203b063 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:51.859379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cde4e-68b8-4773-9cec-8ef78203b063 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:51.860090 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cde4e-68b8-4773-9cec-8ef78203b063 None admin] RBAC: Adding query filter params (type=image) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:51.860401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cde4e-68b8-4773-9cec-8ef78203b063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:51.862249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cde4e-68b8-4773-9cec-8ef78203b063 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:51.867537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 503/997] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:51.887865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:51.891267 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd1fe7-9cf9-4b56-aedc-0f192ea8ea11 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:51.929994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638684a8-f0d8-46a0-a12e-e4f078fe8147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:51.931013 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638684a8-f0d8-46a0-a12e-e4f078fe8147 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:51.931510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638684a8-f0d8-46a0-a12e-e4f078fe8147 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:51.932017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638684a8-f0d8-46a0-a12e-e4f078fe8147 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:51.932986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638684a8-f0d8-46a0-a12e-e4f078fe8147 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:51.933503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638684a8-f0d8-46a0-a12e-e4f078fe8147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:51.935086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638684a8-f0d8-46a0-a12e-e4f078fe8147 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:51.941858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 495/998] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:51.947722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cde4e-68b8-4773-9cec-8ef78203b063 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:52.034334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:52.067772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cde4e-68b8-4773-9cec-8ef78203b063 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:52.102879 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0974ce8-99c0-429a-aa72-40b4beecba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:52.103543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0974ce8-99c0-429a-aa72-40b4beecba6a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:52.103823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0974ce8-99c0-429a-aa72-40b4beecba6a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:52.104094 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0974ce8-99c0-429a-aa72-40b4beecba6a None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:52.104833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0974ce8-99c0-429a-aa72-40b4beecba6a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:52.105140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0974ce8-99c0-429a-aa72-40b4beecba6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:52.106499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0974ce8-99c0-429a-aa72-40b4beecba6a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:52.116659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 504/999] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:52.118163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638684a8-f0d8-46a0-a12e-e4f078fe8147 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:52.221178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638684a8-f0d8-46a0-a12e-e4f078fe8147 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:52.256173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0790a9-c8c5-4119-820a-2030b91bdcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:52.257117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0790a9-c8c5-4119-820a-2030b91bdcbe None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:52.257594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0790a9-c8c5-4119-820a-2030b91bdcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:52.258059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0790a9-c8c5-4119-820a-2030b91bdcbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:52.259018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0790a9-c8c5-4119-820a-2030b91bdcbe None admin] RBAC: Adding query filter params (interface=public, service_id=9307c794cc1f4e788fc7631e627f0a9f, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:52.259496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0790a9-c8c5-4119-820a-2030b91bdcbe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:52.261040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0790a9-c8c5-4119-820a-2030b91bdcbe None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:52.266765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 496/1000] 10.222.0.69 () {56 vars in 1251 bytes} [Fri Jun 24 10:23:52 2022] GET /identity/v3/endpoints?service_id=9307c794cc1f4e788fc7631e627f0a9f&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:52.272224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0974ce8-99c0-429a-aa72-40b4beecba6a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:52.378881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0974ce8-99c0-429a-aa72-40b4beecba6a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:52.416105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77d7d2-c994-4439-94af-5b630fb13cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:52.416788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77d7d2-c994-4439-94af-5b630fb13cdc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:52.417085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77d7d2-c994-4439-94af-5b630fb13cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:52.417363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77d7d2-c994-4439-94af-5b630fb13cdc None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:52.418049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77d7d2-c994-4439-94af-5b630fb13cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:52.419415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77d7d2-c994-4439-94af-5b630fb13cdc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:52.426531 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 505/1001] 10.222.0.69 () {56 vars in 1087 bytes} [Fri Jun 24 10:23:52 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:23:53.008425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3308888-e214-47c8-96d7-3f4e4bdbf22b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:53.008867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3308888-e214-47c8-96d7-3f4e4bdbf22b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:53.009042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3308888-e214-47c8-96d7-3f4e4bdbf22b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:53.010653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 497/1002] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:53.016296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d37e-282c-4cc6-9784-bafb65331b7c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:53.016594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d37e-282c-4cc6-9784-bafb65331b7c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:53.016907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d37e-282c-4cc6-9784-bafb65331b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:53.022151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:53.035830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ff85d37e-282c-4cc6-9784-bafb65331b7c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:53.037083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ff85d37e-282c-4cc6-9784-bafb65331b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:53.109772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 506/1003] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:53.133264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3308888-e214-47c8-96d7-3f4e4bdbf22b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:53.164432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:53.245360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3308888-e214-47c8-96d7-3f4e4bdbf22b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:53.280946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60470e27-0872-4db4-a3cd-cbb2beb18a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:53.281673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60470e27-0872-4db4-a3cd-cbb2beb18a99 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:53.282007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60470e27-0872-4db4-a3cd-cbb2beb18a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:53.282283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60470e27-0872-4db4-a3cd-cbb2beb18a99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:53.282942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60470e27-0872-4db4-a3cd-cbb2beb18a99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:53.284300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60470e27-0872-4db4-a3cd-cbb2beb18a99 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-60470e27-0872-4db4-a3cd-cbb2beb18a99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:53.296663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:53.297775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:23:53.298608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:53.301105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 498/1004] 10.222.0.69 () {56 vars in 1126 bytes} [Fri Jun 24 10:23:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:53.303835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d37e-282c-4cc6-9784-bafb65331b7c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:53.409086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d37e-282c-4cc6-9784-bafb65331b7c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:53.444399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:53.448902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a79294-7719-44af-a17a-ab3dec83c4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:53.449582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a79294-7719-44af-a17a-ab3dec83c4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:53.449875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a79294-7719-44af-a17a-ab3dec83c4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:53.450163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a79294-7719-44af-a17a-ab3dec83c4d0 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:53.450885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a79294-7719-44af-a17a-ab3dec83c4d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:53.451283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a79294-7719-44af-a17a-ab3dec83c4d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:53.455368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a79294-7719-44af-a17a-ab3dec83c4d0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:53.464765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 507/1005] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:53.466529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60470e27-0872-4db4-a3cd-cbb2beb18a99 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:53.566188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:53.576699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60470e27-0872-4db4-a3cd-cbb2beb18a99 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:53.610976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06c802-b6e9-404d-b2f8-c0ad9c606f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:53.611641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06c802-b6e9-404d-b2f8-c0ad9c606f4b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:53.611916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06c802-b6e9-404d-b2f8-c0ad9c606f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:53.612190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06c802-b6e9-404d-b2f8-c0ad9c606f4b None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:53.612931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06c802-b6e9-404d-b2f8-c0ad9c606f4b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:53.613235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06c802-b6e9-404d-b2f8-c0ad9c606f4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:53.614584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06c802-b6e9-404d-b2f8-c0ad9c606f4b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:53.619705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 499/1006] 10.222.0.69 () {56 vars in 1122 bytes} [Fri Jun 24 10:23:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:53.625482 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a79294-7719-44af-a17a-ab3dec83c4d0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:53.727360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a79294-7719-44af-a17a-ab3dec83c4d0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:53.761606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437815c-045a-4b8a-aa73-fcbdf247920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:53.762257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437815c-045a-4b8a-aa73-fcbdf247920d None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:53.762534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437815c-045a-4b8a-aa73-fcbdf247920d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:53.762809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437815c-045a-4b8a-aa73-fcbdf247920d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:53.763535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437815c-045a-4b8a-aa73-fcbdf247920d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:53.765182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437815c-045a-4b8a-aa73-fcbdf247920d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:53.788782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 508/1007] 10.222.0.69 () {60 vars in 1144 bytes} [Fri Jun 24 10:23:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:53.870404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851658b-cab4-421e-a4cc-1c21f570958b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:53.870721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851658b-cab4-421e-a4cc-1c21f570958b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:53.870945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851658b-cab4-421e-a4cc-1c21f570958b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:53.872524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 500/1008] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:53.878687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd9e3-a3ee-47b0-a758-fd446f14640a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:53.879319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd9e3-a3ee-47b0-a758-fd446f14640a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:53.879804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd9e3-a3ee-47b0-a758-fd446f14640a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:53.899289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5aecd9e3-a3ee-47b0-a758-fd446f14640a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:53.900902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5aecd9e3-a3ee-47b0-a758-fd446f14640a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:53.960848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:53.984213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 509/1009] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:54.006858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851658b-cab4-421e-a4cc-1c21f570958b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:54.115991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851658b-cab4-421e-a4cc-1c21f570958b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:54.150968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cd743-5ba8-4984-898e-a9e895de409a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:54.151978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cd743-5ba8-4984-898e-a9e895de409a None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:54.152471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cd743-5ba8-4984-898e-a9e895de409a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:54.152981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cd743-5ba8-4984-898e-a9e895de409a None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:54.153921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cd743-5ba8-4984-898e-a9e895de409a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:54.155540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cd743-5ba8-4984-898e-a9e895de409a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:54.175317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 501/1010] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:23:54 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:54.432354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab60255-6b5a-46c2-8ce7-153acdd043eb None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:54.432927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab60255-6b5a-46c2-8ce7-153acdd043eb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:54.432927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab60255-6b5a-46c2-8ce7-153acdd043eb None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:54.434339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 510/1011] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:54.441638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613098c-724c-4481-997f-220d99b5a740 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:54.441638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613098c-724c-4481-997f-220d99b5a740 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:54.441638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613098c-724c-4481-997f-220d99b5a740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:54.458816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8613098c-724c-4481-997f-220d99b5a740 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:54.460015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8613098c-724c-4481-997f-220d99b5a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:54.472091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:54.538172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 502/1012] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:54 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:54.561026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab60255-6b5a-46c2-8ce7-153acdd043eb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:54.662854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab60255-6b5a-46c2-8ce7-153acdd043eb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:54.697720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec6bd2-f1f6-4b4f-8358-c361f0141d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:54.698431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec6bd2-f1f6-4b4f-8358-c361f0141d6d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:54.698710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec6bd2-f1f6-4b4f-8358-c361f0141d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:54.698985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec6bd2-f1f6-4b4f-8358-c361f0141d6d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:54.699636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec6bd2-f1f6-4b4f-8358-c361f0141d6d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:54.701026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec6bd2-f1f6-4b4f-8358-c361f0141d6d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-03ec6bd2-f1f6-4b4f-8358-c361f0141d6d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:54.712967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:54.713922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:23:54.714747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:54.717141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 511/1013] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:54.720448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613098c-724c-4481-997f-220d99b5a740 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:54.825234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613098c-724c-4481-997f-220d99b5a740 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:54.859753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd29675-00e2-49f7-9b16-1dad7df68dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:54.860425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd29675-00e2-49f7-9b16-1dad7df68dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:54.860709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd29675-00e2-49f7-9b16-1dad7df68dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:54.861010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd29675-00e2-49f7-9b16-1dad7df68dd7 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:54.861726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd29675-00e2-49f7-9b16-1dad7df68dd7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:54.862032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd29675-00e2-49f7-9b16-1dad7df68dd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:54.863400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd29675-00e2-49f7-9b16-1dad7df68dd7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:54.868762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 503/1014] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:54.874281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec6bd2-f1f6-4b4f-8358-c361f0141d6d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:54.927843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:54.985432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec6bd2-f1f6-4b4f-8358-c361f0141d6d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:55.019823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffad18-cf27-47e6-b9a2-ef63cf89c1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:55.020490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffad18-cf27-47e6-b9a2-ef63cf89c1cf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:55.020787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffad18-cf27-47e6-b9a2-ef63cf89c1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:55.021070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffad18-cf27-47e6-b9a2-ef63cf89c1cf None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:55.021775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffad18-cf27-47e6-b9a2-ef63cf89c1cf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:55.022078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffad18-cf27-47e6-b9a2-ef63cf89c1cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:55.023501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffad18-cf27-47e6-b9a2-ef63cf89c1cf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:55.028657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 512/1015] 10.222.0.69 () {56 vars in 1112 bytes} [Fri Jun 24 10:23:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:55.034344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd29675-00e2-49f7-9b16-1dad7df68dd7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:55.043688 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:55.138610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd29675-00e2-49f7-9b16-1dad7df68dd7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:55.172472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ae726-eb0b-48af-9463-83202ad02264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:55.173164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ae726-eb0b-48af-9463-83202ad02264 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:55.173443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ae726-eb0b-48af-9463-83202ad02264 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:55.173719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ae726-eb0b-48af-9463-83202ad02264 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:55.174468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ae726-eb0b-48af-9463-83202ad02264 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:55.175824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ae726-eb0b-48af-9463-83202ad02264 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:55.200762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 504/1016] 10.222.0.69 () {60 vars in 1144 bytes} [Fri Jun 24 10:23:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:55.834081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d025fc-faf6-466a-9616-dfd6938a76e2 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:55.834601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d025fc-faf6-466a-9616-dfd6938a76e2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:55.834963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d025fc-faf6-466a-9616-dfd6938a76e2 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:55.837294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 513/1017] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:55.843163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585f062-20e2-4747-b8a1-c248f17d0122 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:55.843604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585f062-20e2-4747-b8a1-c248f17d0122 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:55.843858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585f062-20e2-4747-b8a1-c248f17d0122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:55.862356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c585f062-20e2-4747-b8a1-c248f17d0122 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:55.863547 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c585f062-20e2-4747-b8a1-c248f17d0122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:55.924428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:55.943199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 505/1018] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:55.966778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d025fc-faf6-466a-9616-dfd6938a76e2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:56.074813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d025fc-faf6-466a-9616-dfd6938a76e2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:56.085519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:56.110737 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264f062-63c3-4264-8fb8-33ba924c7776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:56.111519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264f062-63c3-4264-8fb8-33ba924c7776 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:56.111798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264f062-63c3-4264-8fb8-33ba924c7776 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:56.112067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264f062-63c3-4264-8fb8-33ba924c7776 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:56.113254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264f062-63c3-4264-8fb8-33ba924c7776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:56.114694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264f062-63c3-4264-8fb8-33ba924c7776 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:56.116495 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 514/1019] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:56.122310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585f062-20e2-4747-b8a1-c248f17d0122 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:56.222541 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585f062-20e2-4747-b8a1-c248f17d0122 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:56.257650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516f906-0e6b-4507-8cbe-4ae0d70c483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:56.258312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516f906-0e6b-4507-8cbe-4ae0d70c483d None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:56.258590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516f906-0e6b-4507-8cbe-4ae0d70c483d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:56.258862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516f906-0e6b-4507-8cbe-4ae0d70c483d None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:56.259571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516f906-0e6b-4507-8cbe-4ae0d70c483d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:56.261042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516f906-0e6b-4507-8cbe-4ae0d70c483d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:56.293108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 506/1020] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:23:56 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:56.293302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a67a2b-382d-4db1-a23b-670638e18910 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:56.293384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a67a2b-382d-4db1-a23b-670638e18910 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:56.293780 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a67a2b-382d-4db1-a23b-670638e18910 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:56.295115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 515/1021] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:56.301160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d34592-4a68-4309-8456-79f2beb605b9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:56.301466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d34592-4a68-4309-8456-79f2beb605b9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:56.301727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d34592-4a68-4309-8456-79f2beb605b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:56.321205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-59d34592-4a68-4309-8456-79f2beb605b9 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:56.322422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-59d34592-4a68-4309-8456-79f2beb605b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:56.364300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:56.406808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 507/1022] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:56.430077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a67a2b-382d-4db1-a23b-670638e18910 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:56.546118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a67a2b-382d-4db1-a23b-670638e18910 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:56.581379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438c33-bb7a-4941-8760-fa037a5de89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:56.582166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438c33-bb7a-4941-8760-fa037a5de89d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:56.582445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438c33-bb7a-4941-8760-fa037a5de89d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:56.582716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438c33-bb7a-4941-8760-fa037a5de89d None admin] PATH_INFO: `/v3/services/image` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:56.583363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438c33-bb7a-4941-8760-fa037a5de89d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:56.584721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438c33-bb7a-4941-8760-fa037a5de89d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a9438c33-bb7a-4941-8760-fa037a5de89d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:56.596205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:56.597243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:23:56.598151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 516/1023] 10.222.0.69 () {56 vars in 1105 bytes} [Fri Jun 24 10:23:56 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:56.605299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d34592-4a68-4309-8456-79f2beb605b9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:56.712149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d34592-4a68-4309-8456-79f2beb605b9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:56.747252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb01889-f59d-4925-8906-ea1a7a1bde42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:56.747925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb01889-f59d-4925-8906-ea1a7a1bde42 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:56.748222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb01889-f59d-4925-8906-ea1a7a1bde42 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:56.748527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb01889-f59d-4925-8906-ea1a7a1bde42 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:56.749262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb01889-f59d-4925-8906-ea1a7a1bde42 None admin] RBAC: Adding query filter params (name=image) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:56.749573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb01889-f59d-4925-8906-ea1a7a1bde42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:56.750935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb01889-f59d-4925-8906-ea1a7a1bde42 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:56.756441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 508/1024] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:56.761960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438c33-bb7a-4941-8760-fa037a5de89d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:56.808528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:56.868354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438c33-bb7a-4941-8760-fa037a5de89d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:56.904117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be456c7-9b31-4432-bede-dca65fea4ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:56.904856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be456c7-9b31-4432-bede-dca65fea4ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:56.905137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be456c7-9b31-4432-bede-dca65fea4ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:56.905412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be456c7-9b31-4432-bede-dca65fea4ce8 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:56.906123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be456c7-9b31-4432-bede-dca65fea4ce8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:56.906430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be456c7-9b31-4432-bede-dca65fea4ce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:56.907797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be456c7-9b31-4432-bede-dca65fea4ce8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:56.913121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 517/1025] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:56.918841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb01889-f59d-4925-8906-ea1a7a1bde42 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:56.953650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:57.023285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb01889-f59d-4925-8906-ea1a7a1bde42 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:57.057311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d6305-9407-4f03-a52e-c9488e8f5123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:57.057976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d6305-9407-4f03-a52e-c9488e8f5123 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:57.058307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d6305-9407-4f03-a52e-c9488e8f5123 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:57.058589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d6305-9407-4f03-a52e-c9488e8f5123 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:57.059375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d6305-9407-4f03-a52e-c9488e8f5123 None admin] RBAC: Adding query filter params (interface=public, service_id=fe337bd9d7914cd4bb9efa88e619d619, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:57.059687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d6305-9407-4f03-a52e-c9488e8f5123 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:57.061134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d6305-9407-4f03-a52e-c9488e8f5123 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:57.070903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 509/1026] 10.222.0.69 () {56 vars in 1251 bytes} [Fri Jun 24 10:23:56 2022] GET /identity/v3/endpoints?service_id=fe337bd9d7914cd4bb9efa88e619d619&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:57.072120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be456c7-9b31-4432-bede-dca65fea4ce8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:57.172364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be456c7-9b31-4432-bede-dca65fea4ce8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:57.179485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:57.208350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5416fc-0cf8-493b-bde3-52102a0a92f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:57.209038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5416fc-0cf8-493b-bde3-52102a0a92f1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:57.209373 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5416fc-0cf8-493b-bde3-52102a0a92f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:57.209655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5416fc-0cf8-493b-bde3-52102a0a92f1 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:57.210335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5416fc-0cf8-493b-bde3-52102a0a92f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:57.211702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5416fc-0cf8-493b-bde3-52102a0a92f1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:57.218519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 518/1027] 10.222.0.69 () {56 vars in 1087 bytes} [Fri Jun 24 10:23:57 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:23:57.304188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205d6b-92fd-4d53-9ceb-a6ad95b36a0d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:57.304379 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205d6b-92fd-4d53-9ceb-a6ad95b36a0d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:57.304600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205d6b-92fd-4d53-9ceb-a6ad95b36a0d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:57.306250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 510/1028] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:57.311804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bca978-bede-4aba-979d-0de388fecb14 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:57.312114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bca978-bede-4aba-979d-0de388fecb14 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:57.312368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bca978-bede-4aba-979d-0de388fecb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:57.330383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c7bca978-bede-4aba-979d-0de388fecb14 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:57.331918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c7bca978-bede-4aba-979d-0de388fecb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:57.404955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 519/1029] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:57.427986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78205d6b-92fd-4d53-9ceb-a6ad95b36a0d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:57.536311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:57.542537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78205d6b-92fd-4d53-9ceb-a6ad95b36a0d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:57.578715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d4ce5-0937-413f-ab7e-76d24a15f08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:57.579441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d4ce5-0937-413f-ab7e-76d24a15f08b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:57.579719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d4ce5-0937-413f-ab7e-76d24a15f08b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:57.579988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d4ce5-0937-413f-ab7e-76d24a15f08b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:57.590561 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d4ce5-0937-413f-ab7e-76d24a15f08b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ec9d4ce5-0937-413f-ab7e-76d24a15f08b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:23:57.598750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:57.599654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:23:57.600530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:57.604825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 511/1030] 10.222.0.69 () {56 vars in 1120 bytes} [Fri Jun 24 10:23:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:57.605838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bca978-bede-4aba-979d-0de388fecb14 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:57.705163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:57.709382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bca978-bede-4aba-979d-0de388fecb14 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:57.744769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2c4f1-604f-4826-9b08-e7567a900e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:57.745465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2c4f1-604f-4826-9b08-e7567a900e55 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:57.745747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2c4f1-604f-4826-9b08-e7567a900e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:57.746018 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2c4f1-604f-4826-9b08-e7567a900e55 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:57.746767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2c4f1-604f-4826-9b08-e7567a900e55 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:57.747071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2c4f1-604f-4826-9b08-e7567a900e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:57.748607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2c4f1-604f-4826-9b08-e7567a900e55 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:57.756985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 520/1031] 10.222.0.69 () {56 vars in 1115 bytes} [Fri Jun 24 10:23:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:57.762451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d4ce5-0937-413f-ab7e-76d24a15f08b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:57.866297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d4ce5-0937-413f-ab7e-76d24a15f08b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:57.902308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715b361-a794-4b95-a5a0-006b0862fef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:57.903403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715b361-a794-4b95-a5a0-006b0862fef2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:57.903854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715b361-a794-4b95-a5a0-006b0862fef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:57.904349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715b361-a794-4b95-a5a0-006b0862fef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:57.915939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715b361-a794-4b95-a5a0-006b0862fef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:57.917642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715b361-a794-4b95-a5a0-006b0862fef2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b715b361-a794-4b95-a5a0-006b0862fef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:57.935286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:57.936679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:57.937618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:57.937618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:57.937618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:57.937618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:57.943126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 512/1032] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:57.944358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2c4f1-604f-4826-9b08-e7567a900e55 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:58.049198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2c4f1-604f-4826-9b08-e7567a900e55 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:58.083838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0caa9b-86ee-43e1-a9b5-27ab9dc7d197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:58.084786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0caa9b-86ee-43e1-a9b5-27ab9dc7d197 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:58.085266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0caa9b-86ee-43e1-a9b5-27ab9dc7d197 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:58.085712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0caa9b-86ee-43e1-a9b5-27ab9dc7d197 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:58.086602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0caa9b-86ee-43e1-a9b5-27ab9dc7d197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:58.087075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0caa9b-86ee-43e1-a9b5-27ab9dc7d197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:58.088616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0caa9b-86ee-43e1-a9b5-27ab9dc7d197 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:58.104994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 521/1033] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:58.105985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715b361-a794-4b95-a5a0-006b0862fef2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:58.119799 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:58.229586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715b361-a794-4b95-a5a0-006b0862fef2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:58.263181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe86ea-a3d9-4998-badf-a510425d8d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:58.264040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe86ea-a3d9-4998-badf-a510425d8d34 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:58.264321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe86ea-a3d9-4998-badf-a510425d8d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:58.264569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe86ea-a3d9-4998-badf-a510425d8d34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:58.272478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe86ea-a3d9-4998-badf-a510425d8d34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:58.273937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe86ea-a3d9-4998-badf-a510425d8d34 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a3fe86ea-a3d9-4998-badf-a510425d8d34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:23:58.287855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:58.288805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:58.293685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 513/1034] 10.222.0.69 () {56 vars in 1093 bytes} [Fri Jun 24 10:23:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:58.296712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0caa9b-86ee-43e1-a9b5-27ab9dc7d197 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:58.330556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:58.360864 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0925063-6708-48e0-bcea-a5e1106183d4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:58.361111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0925063-6708-48e0-bcea-a5e1106183d4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:58.361405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0925063-6708-48e0-bcea-a5e1106183d4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:58.362945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 514/1035] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:23:58.367790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925893cf-9107-4f4d-85ca-4f4d2c87acb1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:58.368082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925893cf-9107-4f4d-85ca-4f4d2c87acb1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:58.368361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925893cf-9107-4f4d-85ca-4f4d2c87acb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:58.385467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-925893cf-9107-4f4d-85ca-4f4d2c87acb1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:58.386469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-925893cf-9107-4f4d-85ca-4f4d2c87acb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:58.413449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0caa9b-86ee-43e1-a9b5-27ab9dc7d197 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:58.453760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74500ad3-c884-48f1-b40c-c59e0679aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:58.454845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74500ad3-c884-48f1-b40c-c59e0679aa0a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:58.455347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74500ad3-c884-48f1-b40c-c59e0679aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:58.456281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74500ad3-c884-48f1-b40c-c59e0679aa0a None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:58.457387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74500ad3-c884-48f1-b40c-c59e0679aa0a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:58.457901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74500ad3-c884-48f1-b40c-c59e0679aa0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:58.458973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:58.460321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74500ad3-c884-48f1-b40c-c59e0679aa0a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:58.461830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 515/1036] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:23:58.480157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 522/1037] 10.222.0.69 () {56 vars in 1133 bytes} [Fri Jun 24 10:23:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:58.486216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925893cf-9107-4f4d-85ca-4f4d2c87acb1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:58.486761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74500ad3-c884-48f1-b40c-c59e0679aa0a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:58.570918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:58.591190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925893cf-9107-4f4d-85ca-4f4d2c87acb1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:58.592781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74500ad3-c884-48f1-b40c-c59e0679aa0a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:58.626071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485080b-05fe-4ec9-90ff-4e51aa15e9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:58.626721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485080b-05fe-4ec9-90ff-4e51aa15e9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:58.626998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485080b-05fe-4ec9-90ff-4e51aa15e9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:58.627095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568a497-8da3-45f4-8969-5948f064ce41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:58.627261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485080b-05fe-4ec9-90ff-4e51aa15e9b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:58.627830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568a497-8da3-45f4-8969-5948f064ce41 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:58.628181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568a497-8da3-45f4-8969-5948f064ce41 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:58.628388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568a497-8da3-45f4-8969-5948f064ce41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:58.629570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568a497-8da3-45f4-8969-5948f064ce41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:58.631982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568a497-8da3-45f4-8969-5948f064ce41 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:58.638216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 523/1038] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:58.639164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568a497-8da3-45f4-8969-5948f064ce41 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:58.695863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:58.698834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485080b-05fe-4ec9-90ff-4e51aa15e9b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:58.700630 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485080b-05fe-4ec9-90ff-4e51aa15e9b8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:58.745169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568a497-8da3-45f4-8969-5948f064ce41 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:58.783204 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6714b7-8172-432d-ab85-2cfaad656843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:58.783971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6714b7-8172-432d-ab85-2cfaad656843 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:58.784258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6714b7-8172-432d-ab85-2cfaad656843 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:58.784535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6714b7-8172-432d-ab85-2cfaad656843 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:58.785289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6714b7-8172-432d-ab85-2cfaad656843 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:58.786721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6714b7-8172-432d-ab85-2cfaad656843 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:58.807656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 516/1039] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:23:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:23:58.820773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 524/1040] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:23:58 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:58.822376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485080b-05fe-4ec9-90ff-4e51aa15e9b8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:58.928855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:58.932592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485080b-05fe-4ec9-90ff-4e51aa15e9b8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:58.970995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43425808-882a-43b1-a645-c10177b3ec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:58.971794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43425808-882a-43b1-a645-c10177b3ec63 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:58.972070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43425808-882a-43b1-a645-c10177b3ec63 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:58.972340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43425808-882a-43b1-a645-c10177b3ec63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:58.983224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43425808-882a-43b1-a645-c10177b3ec63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:58.984658 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43425808-882a-43b1-a645-c10177b3ec63 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-43425808-882a-43b1-a645-c10177b3ec63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:23:59.002473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:59.003383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:59.004272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:23:59.004272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:23:59.004272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:23:59.004272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:59.004272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 517/1041] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:59.009582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6714b7-8172-432d-ab85-2cfaad656843 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:59.014791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:59.113871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6714b7-8172-432d-ab85-2cfaad656843 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:59.148280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283a042-1bc9-4914-b190-d17da3a72f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:59.148975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283a042-1bc9-4914-b190-d17da3a72f71 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:59.149254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283a042-1bc9-4914-b190-d17da3a72f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:59.149524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283a042-1bc9-4914-b190-d17da3a72f71 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:59.150230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283a042-1bc9-4914-b190-d17da3a72f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:59.150549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283a042-1bc9-4914-b190-d17da3a72f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:59.151900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283a042-1bc9-4914-b190-d17da3a72f71 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:59.164052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 525/1042] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:59.169484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43425808-882a-43b1-a645-c10177b3ec63 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:59.234423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb92d-a466-4a7d-ae43-d408fb7c6c75 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:59.234713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb92d-a466-4a7d-ae43-d408fb7c6c75 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:59.234963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb92d-a466-4a7d-ae43-d408fb7c6c75 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:59.236503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 526/1043] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:23:59.242061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610f8cd-cbbd-4606-806d-4b0470125962 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:59.242380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610f8cd-cbbd-4606-806d-4b0470125962 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:59.242662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610f8cd-cbbd-4606-806d-4b0470125962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:59.259612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6610f8cd-cbbd-4606-806d-4b0470125962 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:59.260858 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6610f8cd-cbbd-4606-806d-4b0470125962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:59.274102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43425808-882a-43b1-a645-c10177b3ec63 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:59.310055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfde86-4a0f-41b2-8257-78abca855bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:59.311471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfde86-4a0f-41b2-8257-78abca855bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:59.311754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfde86-4a0f-41b2-8257-78abca855bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:59.312030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfde86-4a0f-41b2-8257-78abca855bd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:59.319156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bfde86-4a0f-41b2-8257-78abca855bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:59.320872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bfde86-4a0f-41b2-8257-78abca855bd2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:59.322771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:59.333314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 527/1044] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:23:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-22bfde86-4a0f-41b2-8257-78abca855bd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:23:59.335699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:59.336559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:59.338079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:23:59.338079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:23:59.338079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:23:59.338079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:23:59.338079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:23:59.338079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:23:59.338079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:23:59.338079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:59.338079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 518/1045] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:23:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:23:59.342702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610f8cd-cbbd-4606-806d-4b0470125962 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:59.360165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfde86-4a0f-41b2-8257-78abca855bd2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:59.377502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:59.453914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610f8cd-cbbd-4606-806d-4b0470125962 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:59.477415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfde86-4a0f-41b2-8257-78abca855bd2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:59.489921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c32be-c29c-4687-ac5a-4c69e9e973fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:59.490517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c32be-c29c-4687-ac5a-4c69e9e973fc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:59.490799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c32be-c29c-4687-ac5a-4c69e9e973fc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:59.491072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c32be-c29c-4687-ac5a-4c69e9e973fc None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:59.491860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c32be-c29c-4687-ac5a-4c69e9e973fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:59.492168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c32be-c29c-4687-ac5a-4c69e9e973fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:59.493761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c32be-c29c-4687-ac5a-4c69e9e973fc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:59.505396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 528/1046] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:23:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:59.511347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c32be-c29c-4687-ac5a-4c69e9e973fc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:59.513124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde4e7f-60a1-45ab-bc39-dc407176ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:59.514196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde4e7f-60a1-45ab-bc39-dc407176ac75 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:59.514660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde4e7f-60a1-45ab-bc39-dc407176ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:59.515132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde4e7f-60a1-45ab-bc39-dc407176ac75 None admin] PATH_INFO: `/v3/services/image` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:59.515808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde4e7f-60a1-45ab-bc39-dc407176ac75 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:59.517234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde4e7f-60a1-45ab-bc39-dc407176ac75 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-0dde4e7f-60a1-45ab-bc39-dc407176ac75 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:23:59.529558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:23:59.530668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:23:59.531496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 519/1047] 10.222.0.69 () {56 vars in 1105 bytes} [Fri Jun 24 10:23:59 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:23:59.536762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde4e7f-60a1-45ab-bc39-dc407176ac75 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:59.581462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:59.626070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:59.630948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c32be-c29c-4687-ac5a-4c69e9e973fc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:59.651210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde4e7f-60a1-45ab-bc39-dc407176ac75 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:59.668052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b5cea-31d9-4ff5-8aeb-ebfbc1749d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:59.668707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5cea-31d9-4ff5-8aeb-ebfbc1749d12 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:59.669016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5cea-31d9-4ff5-8aeb-ebfbc1749d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:59.669291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5cea-31d9-4ff5-8aeb-ebfbc1749d12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:59.670309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b5cea-31d9-4ff5-8aeb-ebfbc1749d12 None admin] RBAC: Adding query filter params (role.id=f6aab871c6aa416f9b085a37730a5202, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=bec9fee282a64933aa0f6fbdd91a93c4) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:59.670625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b5cea-31d9-4ff5-8aeb-ebfbc1749d12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:59.672105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b5cea-31d9-4ff5-8aeb-ebfbc1749d12 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:59.688772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 529/1048] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:23:59 2022] GET /identity/v3/role_assignments?user.id=bec9fee282a64933aa0f6fbdd91a93c4&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=f6aab871c6aa416f9b085a37730a5202 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:59.699126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2f936-ca76-4b98-90fd-ed5b3b866132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:59.699843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2f936-ca76-4b98-90fd-ed5b3b866132 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:59.700127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2f936-ca76-4b98-90fd-ed5b3b866132 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:59.700405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2f936-ca76-4b98-90fd-ed5b3b866132 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:59.701238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2f936-ca76-4b98-90fd-ed5b3b866132 None admin] RBAC: Adding query filter params (name=image) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:59.701581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2f936-ca76-4b98-90fd-ed5b3b866132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:59.702978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2f936-ca76-4b98-90fd-ed5b3b866132 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:59.709114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 520/1049] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:59.714971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b5cea-31d9-4ff5-8aeb-ebfbc1749d12 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:59.822987 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b5cea-31d9-4ff5-8aeb-ebfbc1749d12 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:59.843540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:59.861423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61905c-f486-4fe8-af6b-03156b3faebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:59.862091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61905c-f486-4fe8-af6b-03156b3faebc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:59.862371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61905c-f486-4fe8-af6b-03156b3faebc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:59.862642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61905c-f486-4fe8-af6b-03156b3faebc None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:59.863413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61905c-f486-4fe8-af6b-03156b3faebc None admin] RBAC: Adding query filter params (type=image) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:59.863719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61905c-f486-4fe8-af6b-03156b3faebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:59.865156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61905c-f486-4fe8-af6b-03156b3faebc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:59.871330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 530/1050] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:23:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:59.876022 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2f936-ca76-4b98-90fd-ed5b3b866132 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:59.979845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2f936-ca76-4b98-90fd-ed5b3b866132 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:00.017094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab03ae-84fa-4b60-9012-31d75c8250d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:00.017769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab03ae-84fa-4b60-9012-31d75c8250d7 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:00.018049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab03ae-84fa-4b60-9012-31d75c8250d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:00.018317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab03ae-84fa-4b60-9012-31d75c8250d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:00.019076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab03ae-84fa-4b60-9012-31d75c8250d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:00.020472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab03ae-84fa-4b60-9012-31d75c8250d7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:00.028767 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:00.042400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 521/1051] 10.222.0.69 () {60 vars in 1144 bytes} [Fri Jun 24 10:23:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:24:00.918916 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1063844-2cd4-4dd6-8646-44655aeed53a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:00.919426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1063844-2cd4-4dd6-8646-44655aeed53a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:00.919504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1063844-2cd4-4dd6-8646-44655aeed53a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:00.921073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 531/1052] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:00.926477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd1400-9598-403e-a86d-9fea3db35522 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:00.926769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd1400-9598-403e-a86d-9fea3db35522 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:00.927041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd1400-9598-403e-a86d-9fea3db35522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:00.944697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c8bd1400-9598-403e-a86d-9fea3db35522 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:00.945894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c8bd1400-9598-403e-a86d-9fea3db35522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:01.018703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 522/1053] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:01.041759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1063844-2cd4-4dd6-8646-44655aeed53a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:01.143733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1063844-2cd4-4dd6-8646-44655aeed53a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:01.179451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72537f5-4f72-403b-a0f4-4e4cc6816ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:01.180167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72537f5-4f72-403b-a0f4-4e4cc6816ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:01.180444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72537f5-4f72-403b-a0f4-4e4cc6816ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:01.180715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72537f5-4f72-403b-a0f4-4e4cc6816ee3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:01.191895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72537f5-4f72-403b-a0f4-4e4cc6816ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f72537f5-4f72-403b-a0f4-4e4cc6816ee3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:01.199336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:01.200285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:01.201284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:01.204849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 532/1054] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:01.207292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd1400-9598-403e-a86d-9fea3db35522 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:01.313260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd1400-9598-403e-a86d-9fea3db35522 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:01.336025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:01.348657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d47af3-d011-427b-a478-dd81de54eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:01.349362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d47af3-d011-427b-a478-dd81de54eccb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:01.349647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d47af3-d011-427b-a478-dd81de54eccb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:01.349922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d47af3-d011-427b-a478-dd81de54eccb None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:01.350676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d47af3-d011-427b-a478-dd81de54eccb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:01.350980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d47af3-d011-427b-a478-dd81de54eccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:01.352350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d47af3-d011-427b-a478-dd81de54eccb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:01.360907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 523/1055] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:24:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:01.366413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72537f5-4f72-403b-a0f4-4e4cc6816ee3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:01.412356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:01.472198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72537f5-4f72-403b-a0f4-4e4cc6816ee3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:01.508154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74592c-23a3-4de2-88dd-c1b93f1da14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:01.509051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74592c-23a3-4de2-88dd-c1b93f1da14a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:01.509301 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74592c-23a3-4de2-88dd-c1b93f1da14a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:01.509604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74592c-23a3-4de2-88dd-c1b93f1da14a None admin] PATH_INFO: `/v3/users/d606fcdc340a4cd7b768ecebc9ebf85c` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:01.523686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74592c-23a3-4de2-88dd-c1b93f1da14a None admin] RBAC: Authorizing `identity:get_user(user_id=d606fcdc340a4cd7b768ecebc9ebf85c)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:01.525171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74592c-23a3-4de2-88dd-c1b93f1da14a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:01.527073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 533/1056] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:01 2022] GET /identity/v3/users/d606fcdc340a4cd7b768ecebc9ebf85c => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:01.532690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d47af3-d011-427b-a478-dd81de54eccb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:01.637871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d47af3-d011-427b-a478-dd81de54eccb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:01.673092 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5edd0d-8b95-4e96-bd39-d102231316c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:01.673738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5edd0d-8b95-4e96-bd39-d102231316c6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:01.674019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5edd0d-8b95-4e96-bd39-d102231316c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:01.674273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5edd0d-8b95-4e96-bd39-d102231316c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:01.695689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2345309-b6eb-4ee9-ad73-13a2351e93bc None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:01.696010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2345309-b6eb-4ee9-ad73-13a2351e93bc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:01.696226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2345309-b6eb-4ee9-ad73-13a2351e93bc None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:01.697917 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 534/1057] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:24:01.703064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42e5d-43c0-46a8-8710-0800468d4a43 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:01.703508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42e5d-43c0-46a8-8710-0800468d4a43 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:01.703795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42e5d-43c0-46a8-8710-0800468d4a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:01.720798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6ca42e5d-43c0-46a8-8710-0800468d4a43 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:01.721924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6ca42e5d-43c0-46a8-8710-0800468d4a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:01.729563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:01.746929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5edd0d-8b95-4e96-bd39-d102231316c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:01.748603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5edd0d-8b95-4e96-bd39-d102231316c6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:01.800066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 535/1058] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:24:01.822931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42e5d-43c0-46a8-8710-0800468d4a43 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:01.854919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:01.857316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 524/1059] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:24:01.858310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5edd0d-8b95-4e96-bd39-d102231316c6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:01.929063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42e5d-43c0-46a8-8710-0800468d4a43 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:01.946627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:01.964026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5edd0d-8b95-4e96-bd39-d102231316c6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:01.973206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64183e-f702-46e2-8e36-fe71f1a837d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:01.974309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64183e-f702-46e2-8e36-fe71f1a837d7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:01.974593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64183e-f702-46e2-8e36-fe71f1a837d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:01.975154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64183e-f702-46e2-8e36-fe71f1a837d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:01.987135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64183e-f702-46e2-8e36-fe71f1a837d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-1d64183e-f702-46e2-8e36-fe71f1a837d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:01.995230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:01.996420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:02.001859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 536/1060] 10.222.0.69 () {56 vars in 1120 bytes} [Fri Jun 24 10:24:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:02.003516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64183e-f702-46e2-8e36-fe71f1a837d7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.006983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71dd61-5370-418d-b880-f6f6d2ba5759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.008648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71dd61-5370-418d-b880-f6f6d2ba5759 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.008963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71dd61-5370-418d-b880-f6f6d2ba5759 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.009241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71dd61-5370-418d-b880-f6f6d2ba5759 None admin] PATH_INFO: `/v3/projects/40a919ef01e24408a577241567900855` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.010417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71dd61-5370-418d-b880-f6f6d2ba5759 None admin] RBAC: Authorizing `identity:get_project(project_id=40a919ef01e24408a577241567900855)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.011849 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71dd61-5370-418d-b880-f6f6d2ba5759 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.014225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 525/1061] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:01 2022] GET /identity/v3/projects/40a919ef01e24408a577241567900855 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:02.020040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71dd61-5370-418d-b880-f6f6d2ba5759 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.101121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:02.111388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64183e-f702-46e2-8e36-fe71f1a837d7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:02.122155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71dd61-5370-418d-b880-f6f6d2ba5759 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:02.144950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a11d39-1312-46f1-a3b8-0c89a2e3ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.145613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a11d39-1312-46f1-a3b8-0c89a2e3ab0f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.145893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a11d39-1312-46f1-a3b8-0c89a2e3ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.146164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a11d39-1312-46f1-a3b8-0c89a2e3ab0f None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.146910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a11d39-1312-46f1-a3b8-0c89a2e3ab0f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:02.147217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a11d39-1312-46f1-a3b8-0c89a2e3ab0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.148558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a11d39-1312-46f1-a3b8-0c89a2e3ab0f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.156628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 537/1062] 10.222.0.69 () {56 vars in 1115 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:02.157870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d9d19-3f09-40a5-866e-b368b0f3657d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.158159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d9d19-3f09-40a5-866e-b368b0f3657d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.158387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4076-33c4-42d9-8d59-5dacf4a97065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.158499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d9d19-3f09-40a5-866e-b368b0f3657d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.159046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4076-33c4-42d9-8d59-5dacf4a97065 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.159329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4076-33c4-42d9-8d59-5dacf4a97065 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.159830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4076-33c4-42d9-8d59-5dacf4a97065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.159928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 538/1063] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:02.160430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c4076-33c4-42d9-8d59-5dacf4a97065 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=40a919ef01e24408a577241567900855, user.id=d606fcdc340a4cd7b768ecebc9ebf85c) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:02.160792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c4076-33c4-42d9-8d59-5dacf4a97065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.162275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c4076-33c4-42d9-8d59-5dacf4a97065 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.164183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d9d19-3f09-40a5-866e-b368b0f3657d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.179548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 526/1064] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/role_assignments?user.id=d606fcdc340a4cd7b768ecebc9ebf85c&scope.project.id=40a919ef01e24408a577241567900855&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:24:02.179548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbdabc-b085-4433-af2b-5cfadc2a08af None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.179548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbdabc-b085-4433-af2b-5cfadc2a08af None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.179548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbdabc-b085-4433-af2b-5cfadc2a08af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.192444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e5bbdabc-b085-4433-af2b-5cfadc2a08af None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:02.193703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e5bbdabc-b085-4433-af2b-5cfadc2a08af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:02.207300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:02.272880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 527/1065] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:02.292696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d9d19-3f09-40a5-866e-b368b0f3657d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:02.294933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbdabc-b085-4433-af2b-5cfadc2a08af None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.330585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18071a0-388a-4043-9da1-fa6625837190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.332291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18071a0-388a-4043-9da1-fa6625837190 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.332291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18071a0-388a-4043-9da1-fa6625837190 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.332291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18071a0-388a-4043-9da1-fa6625837190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.343543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18071a0-388a-4043-9da1-fa6625837190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.345043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18071a0-388a-4043-9da1-fa6625837190 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-e18071a0-388a-4043-9da1-fa6625837190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:24:02.362499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:02.363443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:02.364323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:24:02.364323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:24:02.364323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:02.364323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:02.364323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 539/1066] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:02.369398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18071a0-388a-4043-9da1-fa6625837190 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.373740 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:02.412426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbdabc-b085-4433-af2b-5cfadc2a08af None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:02.455879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bdcc7-ae1c-4a75-a71b-6effb5dbe378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.455879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bdcc7-ae1c-4a75-a71b-6effb5dbe378 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.455879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bdcc7-ae1c-4a75-a71b-6effb5dbe378 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.455879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bdcc7-ae1c-4a75-a71b-6effb5dbe378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.471947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:02.482588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18071a0-388a-4043-9da1-fa6625837190 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:02.520385 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fceecc3-e8e6-4c4d-b2d9-21e1179330fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.521075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fceecc3-e8e6-4c4d-b2d9-21e1179330fc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.521345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fceecc3-e8e6-4c4d-b2d9-21e1179330fc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.521636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fceecc3-e8e6-4c4d-b2d9-21e1179330fc None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.522480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fceecc3-e8e6-4c4d-b2d9-21e1179330fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:02.522795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fceecc3-e8e6-4c4d-b2d9-21e1179330fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.524331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fceecc3-e8e6-4c4d-b2d9-21e1179330fc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.527394 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bdcc7-ae1c-4a75-a71b-6effb5dbe378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.528841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bdcc7-ae1c-4a75-a71b-6effb5dbe378 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.538192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 540/1067] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:24:02.543706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fceecc3-e8e6-4c4d-b2d9-21e1179330fc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.610425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:02.634378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 528/1068] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 10:24:02.640315 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bdcc7-ae1c-4a75-a71b-6effb5dbe378 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.656970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fceecc3-e8e6-4c4d-b2d9-21e1179330fc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:02.690149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:02.691546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6dfac-3276-4409-9578-53f6ef0fd3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.692327 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6dfac-3276-4409-9578-53f6ef0fd3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.692615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6dfac-3276-4409-9578-53f6ef0fd3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.692920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6dfac-3276-4409-9578-53f6ef0fd3d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.700140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6dfac-3276-4409-9578-53f6ef0fd3d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.701680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6dfac-3276-4409-9578-53f6ef0fd3d4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-54a6dfac-3276-4409-9578-53f6ef0fd3d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:24:02.717449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:02.718402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:02.719473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 541/1069] 10.222.0.69 () {56 vars in 1093 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:24:02.724260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6dfac-3276-4409-9578-53f6ef0fd3d4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.748335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bdcc7-ae1c-4a75-a71b-6effb5dbe378 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:02.784134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d6ab4-d30b-4201-8972-292a6c91c7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.784894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6ab4-d30b-4201-8972-292a6c91c7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.785180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6ab4-d30b-4201-8972-292a6c91c7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.785457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6ab4-d30b-4201-8972-292a6c91c7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.796582 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d6ab4-d30b-4201-8972-292a6c91c7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.798278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d6ab4-d30b-4201-8972-292a6c91c7e0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d61d6ab4-d30b-4201-8972-292a6c91c7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:24:02.816292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:02.817220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:02.818194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:24:02.818194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:24:02.818194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:02.818194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:24:02.818194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 529/1070] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:02.823362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d6ab4-d30b-4201-8972-292a6c91c7e0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.833702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6dfac-3276-4409-9578-53f6ef0fd3d4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:02.859627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:02.871512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f7b65-187a-450a-be3b-2aba33b74632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.872395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f7b65-187a-450a-be3b-2aba33b74632 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.872907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f7b65-187a-450a-be3b-2aba33b74632 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.873248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f7b65-187a-450a-be3b-2aba33b74632 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.874291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f7b65-187a-450a-be3b-2aba33b74632 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:02.874612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f7b65-187a-450a-be3b-2aba33b74632 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.876078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f7b65-187a-450a-be3b-2aba33b74632 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.895582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 542/1071] 10.222.0.69 () {56 vars in 1133 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:24:02.901309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f7b65-187a-450a-be3b-2aba33b74632 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:02.925284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:02.929288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d6ab4-d30b-4201-8972-292a6c91c7e0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:02.975168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9929ea-fb42-426f-8d33-c748e4d0460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:02.976194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9929ea-fb42-426f-8d33-c748e4d0460f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.976675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9929ea-fb42-426f-8d33-c748e4d0460f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.977189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9929ea-fb42-426f-8d33-c748e4d0460f None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.978107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9929ea-fb42-426f-8d33-c748e4d0460f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:02.979102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9929ea-fb42-426f-8d33-c748e4d0460f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:02.981149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9929ea-fb42-426f-8d33-c748e4d0460f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:02.996669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 530/1072] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:03.003849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f7b65-187a-450a-be3b-2aba33b74632 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:03.042013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8198961-8509-4842-836a-d56343309241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:03.042745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8198961-8509-4842-836a-d56343309241 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:03.043028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8198961-8509-4842-836a-d56343309241 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:03.043303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8198961-8509-4842-836a-d56343309241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:03.055346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8198961-8509-4842-836a-d56343309241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:03.056859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8198961-8509-4842-836a-d56343309241 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d8198961-8509-4842-836a-d56343309241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:24:03.076166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:03.077188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:03.083220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:24:03.083220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:24:03.083220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:03.083220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:03.083220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 543/1073] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:24:03.083917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9929ea-fb42-426f-8d33-c748e4d0460f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:03.198137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9929ea-fb42-426f-8d33-c748e4d0460f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:03.235187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345be68-b96b-46bc-b5da-6f606f086a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:03.235888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345be68-b96b-46bc-b5da-6f606f086a7a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:03.236166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345be68-b96b-46bc-b5da-6f606f086a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:03.236471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345be68-b96b-46bc-b5da-6f606f086a7a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:03.237242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345be68-b96b-46bc-b5da-6f606f086a7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:03.237543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345be68-b96b-46bc-b5da-6f606f086a7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:03.238938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345be68-b96b-46bc-b5da-6f606f086a7a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:03.253741 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 531/1074] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:24:03.259519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8198961-8509-4842-836a-d56343309241 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:03.296598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:03.364208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8198961-8509-4842-836a-d56343309241 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:03.398746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac14047-6f4b-4add-95cd-9050b4256acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:03.399528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac14047-6f4b-4add-95cd-9050b4256acf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:03.399808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac14047-6f4b-4add-95cd-9050b4256acf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:03.400079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac14047-6f4b-4add-95cd-9050b4256acf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:03.406874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac14047-6f4b-4add-95cd-9050b4256acf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:03.408302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac14047-6f4b-4add-95cd-9050b4256acf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3ac14047-6f4b-4add-95cd-9050b4256acf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:24:03.421891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:03.422816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:03.423715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:24:03.423715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:24:03.423715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:24:03.423715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:24:03.423715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:24:03.423715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:24:03.423715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:24:03.423715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:03.423715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 544/1075] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:24:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:03.429059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345be68-b96b-46bc-b5da-6f606f086a7a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:03.449742 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:03.534878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345be68-b96b-46bc-b5da-6f606f086a7a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:03.569756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73664d9-f59a-47b5-9ac2-ea9c1e523e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:03.570350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73664d9-f59a-47b5-9ac2-ea9c1e523e10 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:03.570640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73664d9-f59a-47b5-9ac2-ea9c1e523e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:03.570913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73664d9-f59a-47b5-9ac2-ea9c1e523e10 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:03.571704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73664d9-f59a-47b5-9ac2-ea9c1e523e10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:03.572017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73664d9-f59a-47b5-9ac2-ea9c1e523e10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:03.573474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73664d9-f59a-47b5-9ac2-ea9c1e523e10 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:03.585863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 532/1076] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:24:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:24:03.591965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac14047-6f4b-4add-95cd-9050b4256acf None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:03.687743 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:03.695703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac14047-6f4b-4add-95cd-9050b4256acf None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:03.730960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9e725-1a0a-4e42-9911-8a2d63a70f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:03.731848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9e725-1a0a-4e42-9911-8a2d63a70f5c None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:03.732127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9e725-1a0a-4e42-9911-8a2d63a70f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:03.732417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9e725-1a0a-4e42-9911-8a2d63a70f5c None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/bec9fee282a64933aa0f6fbdd91a93c4/roles/f6aab871c6aa416f9b085a37730a5202` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:03.734697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9e725-1a0a-4e42-9911-8a2d63a70f5c None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=bec9fee282a64933aa0f6fbdd91a93c4, role_id=f6aab871c6aa416f9b085a37730a5202)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:03.736127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9e725-1a0a-4e42-9911-8a2d63a70f5c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:03.750011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 545/1077] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:24:03 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/bec9fee282a64933aa0f6fbdd91a93c4/roles/f6aab871c6aa416f9b085a37730a5202 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:04.152083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559b6df-31aa-4df4-a487-3ec665146041 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:04.152669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559b6df-31aa-4df4-a487-3ec665146041 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:04.152669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559b6df-31aa-4df4-a487-3ec665146041 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:04.154030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 533/1078] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:04.159580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243ea6b-3db6-4167-bd43-53e9924dcf76 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:04.159979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243ea6b-3db6-4167-bd43-53e9924dcf76 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:04.160143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243ea6b-3db6-4167-bd43-53e9924dcf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:04.178262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3243ea6b-3db6-4167-bd43-53e9924dcf76 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:04.179463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3243ea6b-3db6-4167-bd43-53e9924dcf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:04.252208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 546/1079] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:04.275062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559b6df-31aa-4df4-a487-3ec665146041 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:04.357916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:04.382041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559b6df-31aa-4df4-a487-3ec665146041 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:04.415695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0dc14-fed1-43c9-b351-7e0a68910109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:04.416485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0dc14-fed1-43c9-b351-7e0a68910109 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:04.416786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0dc14-fed1-43c9-b351-7e0a68910109 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:04.417061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0dc14-fed1-43c9-b351-7e0a68910109 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:04.427889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0dc14-fed1-43c9-b351-7e0a68910109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8ab0dc14-fed1-43c9-b351-7e0a68910109 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:04.435373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:04.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:24:04.437335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 534/1080] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:04.443032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243ea6b-3db6-4167-bd43-53e9924dcf76 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:04.530422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:04.547877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243ea6b-3db6-4167-bd43-53e9924dcf76 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:04.582489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad723e1-a040-46a2-b22e-4cb1303b3415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:04.583144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad723e1-a040-46a2-b22e-4cb1303b3415 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:04.583383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad723e1-a040-46a2-b22e-4cb1303b3415 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:04.583682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad723e1-a040-46a2-b22e-4cb1303b3415 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:04.584428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad723e1-a040-46a2-b22e-4cb1303b3415 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:04.584754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad723e1-a040-46a2-b22e-4cb1303b3415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:04.586157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad723e1-a040-46a2-b22e-4cb1303b3415 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:04.594442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 547/1081] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:24:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:04.600396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0dc14-fed1-43c9-b351-7e0a68910109 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:04.703407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0dc14-fed1-43c9-b351-7e0a68910109 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:04.738280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c25bd-f6e2-461e-92e9-04bc6fc3231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:04.739142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c25bd-f6e2-461e-92e9-04bc6fc3231c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:04.739422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c25bd-f6e2-461e-92e9-04bc6fc3231c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:04.739693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c25bd-f6e2-461e-92e9-04bc6fc3231c None admin] PATH_INFO: `/v3/users/d606fcdc340a4cd7b768ecebc9ebf85c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:04.740955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c25bd-f6e2-461e-92e9-04bc6fc3231c None admin] RBAC: Authorizing `identity:get_user(user_id=d606fcdc340a4cd7b768ecebc9ebf85c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:04.742372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c25bd-f6e2-461e-92e9-04bc6fc3231c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:04.744040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 535/1082] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:04 2022] GET /identity/v3/users/d606fcdc340a4cd7b768ecebc9ebf85c => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:04.750033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad723e1-a040-46a2-b22e-4cb1303b3415 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:04.853313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad723e1-a040-46a2-b22e-4cb1303b3415 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:04.887723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25c7ff-706b-4fc8-aa16-a1ec19e066d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:04.888576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25c7ff-706b-4fc8-aa16-a1ec19e066d3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:04.888978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25c7ff-706b-4fc8-aa16-a1ec19e066d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:04.889253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25c7ff-706b-4fc8-aa16-a1ec19e066d3 None admin] PATH_INFO: `/v3/projects/40a919ef01e24408a577241567900855` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:04.890461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25c7ff-706b-4fc8-aa16-a1ec19e066d3 None admin] RBAC: Authorizing `identity:get_project(project_id=40a919ef01e24408a577241567900855)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:04.891918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25c7ff-706b-4fc8-aa16-a1ec19e066d3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:04.893643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 548/1083] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:04 2022] GET /identity/v3/projects/40a919ef01e24408a577241567900855 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:04.899808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c25bd-f6e2-461e-92e9-04bc6fc3231c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:04.908699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:05.000099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c25bd-f6e2-461e-92e9-04bc6fc3231c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:05.034999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254be9b-6a27-4d9a-9747-3c765d2b0d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:05.035867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254be9b-6a27-4d9a-9747-3c765d2b0d79 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:05.036145 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254be9b-6a27-4d9a-9747-3c765d2b0d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:05.036414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254be9b-6a27-4d9a-9747-3c765d2b0d79 None admin] PATH_INFO: `/v3/projects/40a919ef01e24408a577241567900855/users/d606fcdc340a4cd7b768ecebc9ebf85c/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:05.038389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254be9b-6a27-4d9a-9747-3c765d2b0d79 None admin] RBAC: Authorizing `identity:create_grant(project_id=40a919ef01e24408a577241567900855, user_id=d606fcdc340a4cd7b768ecebc9ebf85c, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:05.039793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254be9b-6a27-4d9a-9747-3c765d2b0d79 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:05.055194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 536/1084] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:24:04 2022] PUT /identity/v3/projects/40a919ef01e24408a577241567900855/users/d606fcdc340a4cd7b768ecebc9ebf85c/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:05.756002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991685e7-cb38-47bb-ae31-e9dbc4dca624 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:05.756675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991685e7-cb38-47bb-ae31-e9dbc4dca624 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:05.756675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991685e7-cb38-47bb-ae31-e9dbc4dca624 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:05.758355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 549/1085] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:05.763842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40ef45-7ccb-44d4-b1b0-cb57e9022c0b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:05.764386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40ef45-7ccb-44d4-b1b0-cb57e9022c0b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:05.764386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40ef45-7ccb-44d4-b1b0-cb57e9022c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:05.784968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ec40ef45-7ccb-44d4-b1b0-cb57e9022c0b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:05.786153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ec40ef45-7ccb-44d4-b1b0-cb57e9022c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:05.854017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:05.863133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 537/1086] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:05.887236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991685e7-cb38-47bb-ae31-e9dbc4dca624 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:05.906995 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:06.006629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991685e7-cb38-47bb-ae31-e9dbc4dca624 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.048857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f8527-925b-46b9-9375-4522f1cd3f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.049682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f8527-925b-46b9-9375-4522f1cd3f4e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.049964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f8527-925b-46b9-9375-4522f1cd3f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.050243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f8527-925b-46b9-9375-4522f1cd3f4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.063214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f8527-925b-46b9-9375-4522f1cd3f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-2f5f8527-925b-46b9-9375-4522f1cd3f4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:06.072766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:06.073735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:24:06.074552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:06.075201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 550/1087] 10.222.0.69 () {56 vars in 1120 bytes} [Fri Jun 24 10:24:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:06.081970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40ef45-7ccb-44d4-b1b0-cb57e9022c0b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.191148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40ef45-7ccb-44d4-b1b0-cb57e9022c0b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.228871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71f86c-daee-49a7-80cb-a3ed7305592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.229537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71f86c-daee-49a7-80cb-a3ed7305592a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.229789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71f86c-daee-49a7-80cb-a3ed7305592a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.230089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71f86c-daee-49a7-80cb-a3ed7305592a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.230838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71f86c-daee-49a7-80cb-a3ed7305592a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:06.231143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71f86c-daee-49a7-80cb-a3ed7305592a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.232606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71f86c-daee-49a7-80cb-a3ed7305592a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.241913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 538/1088] 10.222.0.69 () {56 vars in 1115 bytes} [Fri Jun 24 10:24:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:06.247635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f8527-925b-46b9-9375-4522f1cd3f4e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.327135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:06.354516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f8527-925b-46b9-9375-4522f1cd3f4e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.390963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49e674-facf-49af-8e9a-5bb5edfa526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.391748 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49e674-facf-49af-8e9a-5bb5edfa526b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.392033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49e674-facf-49af-8e9a-5bb5edfa526b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.392311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49e674-facf-49af-8e9a-5bb5edfa526b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.402510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49e674-facf-49af-8e9a-5bb5edfa526b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.403996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49e674-facf-49af-8e9a-5bb5edfa526b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-fb49e674-facf-49af-8e9a-5bb5edfa526b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:24:06.421957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:06.422925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:06.428814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:24:06.428814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:24:06.428814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:06.428814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:06.428814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 551/1089] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:06.430189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71f86c-daee-49a7-80cb-a3ed7305592a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.496775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:06.538683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71f86c-daee-49a7-80cb-a3ed7305592a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.573073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40a6f-dc45-4d39-b3c8-94cf3020ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.573731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40a6f-dc45-4d39-b3c8-94cf3020ab64 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.574009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40a6f-dc45-4d39-b3c8-94cf3020ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.574279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40a6f-dc45-4d39-b3c8-94cf3020ab64 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.575003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40a6f-dc45-4d39-b3c8-94cf3020ab64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:06.575305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40a6f-dc45-4d39-b3c8-94cf3020ab64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.576829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40a6f-dc45-4d39-b3c8-94cf3020ab64 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.589598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 539/1090] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:06.595236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49e674-facf-49af-8e9a-5bb5edfa526b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.699076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49e674-facf-49af-8e9a-5bb5edfa526b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.725773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:06.735185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a7152-53c4-445b-a308-c71c3b434cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.735983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a7152-53c4-445b-a308-c71c3b434cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.736265 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a7152-53c4-445b-a308-c71c3b434cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.736542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a7152-53c4-445b-a308-c71c3b434cc0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.744346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a7152-53c4-445b-a308-c71c3b434cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.745843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a7152-53c4-445b-a308-c71c3b434cc0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-fa0a7152-53c4-445b-a308-c71c3b434cc0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:24:06.759944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:06.766377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:06.767326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 552/1091] 10.222.0.69 () {56 vars in 1093 bytes} [Fri Jun 24 10:24:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:06.769319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40a6f-dc45-4d39-b3c8-94cf3020ab64 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.875386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40a6f-dc45-4d39-b3c8-94cf3020ab64 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.892518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:06.912220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e24a3-1a85-4250-bad6-9d09efb1f7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.912930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e24a3-1a85-4250-bad6-9d09efb1f7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.913218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e24a3-1a85-4250-bad6-9d09efb1f7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.913577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e24a3-1a85-4250-bad6-9d09efb1f7c6 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.914434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e24a3-1a85-4250-bad6-9d09efb1f7c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:06.914749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e24a3-1a85-4250-bad6-9d09efb1f7c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.916181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e24a3-1a85-4250-bad6-9d09efb1f7c6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.933995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 540/1092] 10.222.0.69 () {56 vars in 1133 bytes} [Fri Jun 24 10:24:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:06.939889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a7152-53c4-445b-a308-c71c3b434cc0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.045796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a7152-53c4-445b-a308-c71c3b434cc0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.081333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f192add-390a-41ef-8c6b-0004a9b3cef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.082010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f192add-390a-41ef-8c6b-0004a9b3cef8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.082288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f192add-390a-41ef-8c6b-0004a9b3cef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.082586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f192add-390a-41ef-8c6b-0004a9b3cef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.135919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:07.145544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426084b-e18c-45be-be17-cbe7c68e537f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.145830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426084b-e18c-45be-be17-cbe7c68e537f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.146102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426084b-e18c-45be-be17-cbe7c68e537f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.147484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 541/1093] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:07.150891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f192add-390a-41ef-8c6b-0004a9b3cef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:07.152283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034b039-9d12-4000-a0ef-d34251cce501 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.152486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f192add-390a-41ef-8c6b-0004a9b3cef8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:07.152597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034b039-9d12-4000-a0ef-d34251cce501 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.152857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034b039-9d12-4000-a0ef-d34251cce501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.169608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9034b039-9d12-4000-a0ef-d34251cce501 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:07.170844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9034b039-9d12-4000-a0ef-d34251cce501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:07.237947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 542/1094] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:07.256195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 553/1095] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:24:07.257392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034b039-9d12-4000-a0ef-d34251cce501 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.263318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f192add-390a-41ef-8c6b-0004a9b3cef8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.344785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:07.357021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034b039-9d12-4000-a0ef-d34251cce501 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.360133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f192add-390a-41ef-8c6b-0004a9b3cef8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.364222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:07.392998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a232497-6a0d-43d3-bf75-21d6ad581e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.393567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabde480-8538-4e9c-8430-de80ff935295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.393889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a232497-6a0d-43d3-bf75-21d6ad581e57 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.394234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a232497-6a0d-43d3-bf75-21d6ad581e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.394340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabde480-8538-4e9c-8430-de80ff935295 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.394615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a232497-6a0d-43d3-bf75-21d6ad581e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.394706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabde480-8538-4e9c-8430-de80ff935295 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.394881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabde480-8538-4e9c-8430-de80ff935295 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.404974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a232497-6a0d-43d3-bf75-21d6ad581e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:07.405388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabde480-8538-4e9c-8430-de80ff935295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:07.406431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a232497-6a0d-43d3-bf75-21d6ad581e57 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-fabde480-8538-4e9c-8430-de80ff935295 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:07.412907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:07.413833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:07.414651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 554/1096] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:24:07.420039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabde480-8538-4e9c-8430-de80ff935295 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3a232497-6a0d-43d3-bf75-21d6ad581e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:24:07.425044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:07.425905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:07.431617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:24:07.431617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:24:07.431617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:24:07.431617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:24:07.431617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 543/1097] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:07.431617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a232497-6a0d-43d3-bf75-21d6ad581e57 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.523862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabde480-8538-4e9c-8430-de80ff935295 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.532791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a232497-6a0d-43d3-bf75-21d6ad581e57 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.556460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c67f5-e3b3-4cd4-b54a-e75aeae9399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.557152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c67f5-e3b3-4cd4-b54a-e75aeae9399c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.557448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c67f5-e3b3-4cd4-b54a-e75aeae9399c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.557726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c67f5-e3b3-4cd4-b54a-e75aeae9399c None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.558477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c67f5-e3b3-4cd4-b54a-e75aeae9399c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:07.558785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c67f5-e3b3-4cd4-b54a-e75aeae9399c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:07.560156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c67f5-e3b3-4cd4-b54a-e75aeae9399c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:07.565730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cbfb9-caae-4b43-9a00-f5f5b5a02f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.566719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cbfb9-caae-4b43-9a00-f5f5b5a02f37 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.566997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cbfb9-caae-4b43-9a00-f5f5b5a02f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.567263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cbfb9-caae-4b43-9a00-f5f5b5a02f37 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.567972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cbfb9-caae-4b43-9a00-f5f5b5a02f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:07.568277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cbfb9-caae-4b43-9a00-f5f5b5a02f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:07.568397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 555/1098] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:24:07.573307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cbfb9-caae-4b43-9a00-f5f5b5a02f37 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:07.574060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c67f5-e3b3-4cd4-b54a-e75aeae9399c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.585414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:07.585566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 544/1099] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:07.591370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cbfb9-caae-4b43-9a00-f5f5b5a02f37 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.609914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:07.673624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c67f5-e3b3-4cd4-b54a-e75aeae9399c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.691594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cbfb9-caae-4b43-9a00-f5f5b5a02f37 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.707515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd64911-c1cf-454c-809f-b11a79350127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.708395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd64911-c1cf-454c-809f-b11a79350127 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.708678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd64911-c1cf-454c-809f-b11a79350127 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.708996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd64911-c1cf-454c-809f-b11a79350127 None admin] PATH_INFO: `/v3/users/d606fcdc340a4cd7b768ecebc9ebf85c` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.710187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd64911-c1cf-454c-809f-b11a79350127 None admin] RBAC: Authorizing `identity:get_user(user_id=d606fcdc340a4cd7b768ecebc9ebf85c)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:07.711629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd64911-c1cf-454c-809f-b11a79350127 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:07.713272 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 556/1100] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/users/d606fcdc340a4cd7b768ecebc9ebf85c => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:07.719232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd64911-c1cf-454c-809f-b11a79350127 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.725026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55626064-5213-4412-81b2-37e02289aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.725818 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626064-5213-4412-81b2-37e02289aa75 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.726096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626064-5213-4412-81b2-37e02289aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.726365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626064-5213-4412-81b2-37e02289aa75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.733570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55626064-5213-4412-81b2-37e02289aa75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:07.735030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55626064-5213-4412-81b2-37e02289aa75 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-55626064-5213-4412-81b2-37e02289aa75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:24:07.749308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:07.750204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:07.751088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:24:07.751088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:24:07.751088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:24:07.751088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:24:07.751088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:24:07.751088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:24:07.751088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:24:07.751088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:24:07.751088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 545/1101] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:24:07.756294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55626064-5213-4412-81b2-37e02289aa75 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.796171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:07.819032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd64911-c1cf-454c-809f-b11a79350127 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.834598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:07.854043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8aca14-2315-47ff-9f3f-26ed15a7b850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.854670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8aca14-2315-47ff-9f3f-26ed15a7b850 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.854951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8aca14-2315-47ff-9f3f-26ed15a7b850 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.855231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8aca14-2315-47ff-9f3f-26ed15a7b850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.856978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55626064-5213-4412-81b2-37e02289aa75 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.890956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2c8dc-b050-4e7f-99ca-a41b9f6ce17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.891666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2c8dc-b050-4e7f-99ca-a41b9f6ce17f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.891953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2c8dc-b050-4e7f-99ca-a41b9f6ce17f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.892228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2c8dc-b050-4e7f-99ca-a41b9f6ce17f None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.893051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2c8dc-b050-4e7f-99ca-a41b9f6ce17f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:07.893363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2c8dc-b050-4e7f-99ca-a41b9f6ce17f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:07.894805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2c8dc-b050-4e7f-99ca-a41b9f6ce17f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:07.906583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 546/1102] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:24:07.912683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2c8dc-b050-4e7f-99ca-a41b9f6ce17f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.923516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8aca14-2315-47ff-9f3f-26ed15a7b850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:07.925070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8aca14-2315-47ff-9f3f-26ed15a7b850 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:07.999205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:08.013456 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2c8dc-b050-4e7f-99ca-a41b9f6ce17f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:08.022818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 557/1103] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 10:24:08.028763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8aca14-2315-47ff-9f3f-26ed15a7b850 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:08.046737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10c5c8-242f-410e-9e39-1c4be567eaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:08.047392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c5c8-242f-410e-9e39-1c4be567eaec None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:08.047672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c5c8-242f-410e-9e39-1c4be567eaec None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:08.047932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c5c8-242f-410e-9e39-1c4be567eaec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:08.048762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10c5c8-242f-410e-9e39-1c4be567eaec None admin] RBAC: Adding query filter params (role.id=f6aab871c6aa416f9b085a37730a5202, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=bec9fee282a64933aa0f6fbdd91a93c4) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:08.049108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10c5c8-242f-410e-9e39-1c4be567eaec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:08.050539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10c5c8-242f-410e-9e39-1c4be567eaec None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:08.056489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 547/1104] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/role_assignments?user.id=bec9fee282a64933aa0f6fbdd91a93c4&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=f6aab871c6aa416f9b085a37730a5202 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:24:08.131324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8aca14-2315-47ff-9f3f-26ed15a7b850 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:08.169006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23ab00-5ef5-45bb-bbd4-be8798b45591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:08.169880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23ab00-5ef5-45bb-bbd4-be8798b45591 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:08.170211 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23ab00-5ef5-45bb-bbd4-be8798b45591 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:08.170475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23ab00-5ef5-45bb-bbd4-be8798b45591 None admin] PATH_INFO: `/v3/projects/40a919ef01e24408a577241567900855` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:08.171751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23ab00-5ef5-45bb-bbd4-be8798b45591 None admin] RBAC: Authorizing `identity:get_project(project_id=40a919ef01e24408a577241567900855)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:08.173238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23ab00-5ef5-45bb-bbd4-be8798b45591 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:08.175267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 558/1105] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:08 2022] GET /identity/v3/projects/40a919ef01e24408a577241567900855 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:08.181408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10c5c8-242f-410e-9e39-1c4be567eaec None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:08.184684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:08.294182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10c5c8-242f-410e-9e39-1c4be567eaec None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:08.328019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492162b6-1703-4844-8f73-127561abb4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:08.328674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492162b6-1703-4844-8f73-127561abb4ed None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:08.328989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492162b6-1703-4844-8f73-127561abb4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:08.329268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492162b6-1703-4844-8f73-127561abb4ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:08.330105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492162b6-1703-4844-8f73-127561abb4ed None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=40a919ef01e24408a577241567900855, user.id=d606fcdc340a4cd7b768ecebc9ebf85c) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:08.330417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492162b6-1703-4844-8f73-127561abb4ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:08.332149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492162b6-1703-4844-8f73-127561abb4ed None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:08.338138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 548/1106] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:24:08 2022] GET /identity/v3/role_assignments?user.id=d606fcdc340a4cd7b768ecebc9ebf85c&scope.project.id=40a919ef01e24408a577241567900855&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:10.357396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3b501-6d48-496a-9b4c-6c9319d7d317 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.358231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3b501-6d48-496a-9b4c-6c9319d7d317 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.358477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3b501-6d48-496a-9b4c-6c9319d7d317 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.360694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 559/1107] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:10.368577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c04d8-3fe5-4015-ab30-e33f087b8099 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.369038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c04d8-3fe5-4015-ab30-e33f087b8099 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.369126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c04d8-3fe5-4015-ab30-e33f087b8099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.386749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-742c04d8-3fe5-4015-ab30-e33f087b8099 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:10.387920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-742c04d8-3fe5-4015-ab30-e33f087b8099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:10.458057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:10.466840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 549/1108] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:24:10.492439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3b501-6d48-496a-9b4c-6c9319d7d317 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:10.547035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:10.605989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3b501-6d48-496a-9b4c-6c9319d7d317 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:10.640489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64578ca7-2181-460b-9468-b5797e8b5129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:10.641237 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64578ca7-2181-460b-9468-b5797e8b5129 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.641518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64578ca7-2181-460b-9468-b5797e8b5129 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.641794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64578ca7-2181-460b-9468-b5797e8b5129 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.642966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64578ca7-2181-460b-9468-b5797e8b5129 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:10.644389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64578ca7-2181-460b-9468-b5797e8b5129 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:10.646004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 560/1109] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:24:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:10.651970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c04d8-3fe5-4015-ab30-e33f087b8099 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:10.749876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c04d8-3fe5-4015-ab30-e33f087b8099 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:10.784634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5215910-4bb6-4ac7-89df-2400005df2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:10.785309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5215910-4bb6-4ac7-89df-2400005df2bd None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.785586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5215910-4bb6-4ac7-89df-2400005df2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.785860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5215910-4bb6-4ac7-89df-2400005df2bd None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.786562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5215910-4bb6-4ac7-89df-2400005df2bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:10.787972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5215910-4bb6-4ac7-89df-2400005df2bd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:10.817636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 550/1110] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:24:10 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:24:12.861957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a3fda-827d-45b3-9b5e-6418bb5e285d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:12.862497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a3fda-827d-45b3-9b5e-6418bb5e285d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:12.862497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a3fda-827d-45b3-9b5e-6418bb5e285d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:12.863778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 561/1111] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:12.870140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416afe3-6508-486b-854c-e1f46ad4cb4f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:12.870837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416afe3-6508-486b-854c-e1f46ad4cb4f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:12.870972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416afe3-6508-486b-854c-e1f46ad4cb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:12.891038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f416afe3-6508-486b-854c-e1f46ad4cb4f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:12.892233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f416afe3-6508-486b-854c-e1f46ad4cb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:12.946350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:12.975608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 551/1112] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:13.001662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a3fda-827d-45b3-9b5e-6418bb5e285d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:13.107380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a3fda-827d-45b3-9b5e-6418bb5e285d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:13.120452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:13.142500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1add9-a358-42ec-9cd0-7591674b6ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:13.143333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1add9-a358-42ec-9cd0-7591674b6ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:13.143607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1add9-a358-42ec-9cd0-7591674b6ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:13.143872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1add9-a358-42ec-9cd0-7591674b6ad8 None admin] PATH_INFO: `/v3/roles/bde14daa0b464f6f9622e4d84e28e7d1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:13.145404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1add9-a358-42ec-9cd0-7591674b6ad8 None admin] RBAC: Authorizing `identity:get_role(role_id=bde14daa0b464f6f9622e4d84e28e7d1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:13.146771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1add9-a358-42ec-9cd0-7591674b6ad8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:13.148121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 562/1113] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:13 2022] GET /identity/v3/roles/bde14daa0b464f6f9622e4d84e28e7d1 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:13.154139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416afe3-6508-486b-854c-e1f46ad4cb4f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:13.258154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416afe3-6508-486b-854c-e1f46ad4cb4f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:13.295679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9342bd3-279e-447b-a2ae-f52fd1ff6849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:13.296549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9342bd3-279e-447b-a2ae-f52fd1ff6849 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:13.296855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9342bd3-279e-447b-a2ae-f52fd1ff6849 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:13.297115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9342bd3-279e-447b-a2ae-f52fd1ff6849 None admin] PATH_INFO: `/v3/users/0ddd4ee440f94d37b06ac5253f07c951` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:13.311227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9342bd3-279e-447b-a2ae-f52fd1ff6849 None admin] RBAC: Authorizing `identity:get_user(user_id=0ddd4ee440f94d37b06ac5253f07c951)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:13.312657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9342bd3-279e-447b-a2ae-f52fd1ff6849 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:13.314344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 552/1114] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:13 2022] GET /identity/v3/users/0ddd4ee440f94d37b06ac5253f07c951 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:13.321079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1add9-a358-42ec-9cd0-7591674b6ad8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:13.432410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1add9-a358-42ec-9cd0-7591674b6ad8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:13.472230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb43782-b4ff-49d6-afce-5ced1a0ae2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:13.472909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb43782-b4ff-49d6-afce-5ced1a0ae2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:13.473192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb43782-b4ff-49d6-afce-5ced1a0ae2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:13.473468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb43782-b4ff-49d6-afce-5ced1a0ae2e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:13.522400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:13.546628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb43782-b4ff-49d6-afce-5ced1a0ae2e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:13.548100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb43782-b4ff-49d6-afce-5ced1a0ae2e6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:13.652812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 563/1115] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:24:13.659429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9342bd3-279e-447b-a2ae-f52fd1ff6849 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:13.706503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:13.768577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9342bd3-279e-447b-a2ae-f52fd1ff6849 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:13.806791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9e5ae-3105-40c4-8960-7318c4aa5a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:13.807631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9e5ae-3105-40c4-8960-7318c4aa5a0e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:13.807907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9e5ae-3105-40c4-8960-7318c4aa5a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:13.808171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9e5ae-3105-40c4-8960-7318c4aa5a0e None admin] PATH_INFO: `/v3/projects/40a919ef01e24408a577241567900855` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:13.809389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9e5ae-3105-40c4-8960-7318c4aa5a0e None admin] RBAC: Authorizing `identity:get_project(project_id=40a919ef01e24408a577241567900855)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:13.810904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9e5ae-3105-40c4-8960-7318c4aa5a0e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:13.812542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 553/1116] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:13 2022] GET /identity/v3/projects/40a919ef01e24408a577241567900855 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:13.819654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb43782-b4ff-49d6-afce-5ced1a0ae2e6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:13.902873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:13.923349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb43782-b4ff-49d6-afce-5ced1a0ae2e6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:13.959332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680170c-83f2-48dd-830e-92e98007ee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:13.959907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680170c-83f2-48dd-830e-92e98007ee95 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:13.960191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680170c-83f2-48dd-830e-92e98007ee95 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:13.960464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680170c-83f2-48dd-830e-92e98007ee95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:13.961329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680170c-83f2-48dd-830e-92e98007ee95 None admin] RBAC: Adding query filter params (role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=40a919ef01e24408a577241567900855, user.id=0ddd4ee440f94d37b06ac5253f07c951) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:13.961641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680170c-83f2-48dd-830e-92e98007ee95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:13.963076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680170c-83f2-48dd-830e-92e98007ee95 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:13.968691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 564/1117] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:24:13 2022] GET /identity/v3/role_assignments?user.id=0ddd4ee440f94d37b06ac5253f07c951&scope.project.id=40a919ef01e24408a577241567900855&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:15.937806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbccff-287c-4009-a614-fb2ebfffebe4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:15.938675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbccff-287c-4009-a614-fb2ebfffebe4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:15.938675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbccff-287c-4009-a614-fb2ebfffebe4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:15.940858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 554/1118] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:15.949164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335dfe1-b865-4ef3-b762-02db1e60ee35 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:15.949614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335dfe1-b865-4ef3-b762-02db1e60ee35 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:15.949712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335dfe1-b865-4ef3-b762-02db1e60ee35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:15.967570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d335dfe1-b865-4ef3-b762-02db1e60ee35 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:15.968776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d335dfe1-b865-4ef3-b762-02db1e60ee35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:16.037633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 565/1119] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:24:16.063310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbccff-287c-4009-a614-fb2ebfffebe4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:16.173651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbccff-287c-4009-a614-fb2ebfffebe4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:16.182844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:16.216174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39274279-acdc-41ae-be4e-29cd1b3773ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:16.217183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274279-acdc-41ae-be4e-29cd1b3773ac None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:16.217318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274279-acdc-41ae-be4e-29cd1b3773ac None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:16.217590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274279-acdc-41ae-be4e-29cd1b3773ac None admin] PATH_INFO: `/v3/roles/bde14daa0b464f6f9622e4d84e28e7d1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:16.219079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39274279-acdc-41ae-be4e-29cd1b3773ac None admin] RBAC: Authorizing `identity:get_role(role_id=bde14daa0b464f6f9622e4d84e28e7d1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:16.220429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39274279-acdc-41ae-be4e-29cd1b3773ac None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:16.221827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 555/1120] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:16 2022] GET /identity/v3/roles/bde14daa0b464f6f9622e4d84e28e7d1 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:16.227864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335dfe1-b865-4ef3-b762-02db1e60ee35 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:16.307566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:16.334328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335dfe1-b865-4ef3-b762-02db1e60ee35 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:16.368661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7aaca-2ede-4761-b29f-e975b1805db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:16.369575 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7aaca-2ede-4761-b29f-e975b1805db8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:16.369853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7aaca-2ede-4761-b29f-e975b1805db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:16.370124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7aaca-2ede-4761-b29f-e975b1805db8 None admin] PATH_INFO: `/v3/users/0ddd4ee440f94d37b06ac5253f07c951` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:16.371410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7aaca-2ede-4761-b29f-e975b1805db8 None admin] RBAC: Authorizing `identity:get_user(user_id=0ddd4ee440f94d37b06ac5253f07c951)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:16.372869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7aaca-2ede-4761-b29f-e975b1805db8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:16.374575 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 566/1121] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:16 2022] GET /identity/v3/users/0ddd4ee440f94d37b06ac5253f07c951 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:16.383317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39274279-acdc-41ae-be4e-29cd1b3773ac None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:16.492978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39274279-acdc-41ae-be4e-29cd1b3773ac None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:16.530356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aeecd7-b2b3-4df9-8a72-114abfa1fc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:16.531205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aeecd7-b2b3-4df9-8a72-114abfa1fc2d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:16.531488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aeecd7-b2b3-4df9-8a72-114abfa1fc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:16.531762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aeecd7-b2b3-4df9-8a72-114abfa1fc2d None admin] PATH_INFO: `/v3/projects/40a919ef01e24408a577241567900855` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:16.533013 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aeecd7-b2b3-4df9-8a72-114abfa1fc2d None admin] RBAC: Authorizing `identity:get_project(project_id=40a919ef01e24408a577241567900855)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:16.534441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aeecd7-b2b3-4df9-8a72-114abfa1fc2d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:16.536151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 556/1122] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:16 2022] GET /identity/v3/projects/40a919ef01e24408a577241567900855 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:16.542243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7aaca-2ede-4761-b29f-e975b1805db8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:16.648127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7aaca-2ede-4761-b29f-e975b1805db8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:16.684407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d396624-097c-48a3-bcca-55df8a35be87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:16.685311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d396624-097c-48a3-bcca-55df8a35be87 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:16.685669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d396624-097c-48a3-bcca-55df8a35be87 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:16.685945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d396624-097c-48a3-bcca-55df8a35be87 None admin] PATH_INFO: `/v3/projects/40a919ef01e24408a577241567900855/users/0ddd4ee440f94d37b06ac5253f07c951/roles/bde14daa0b464f6f9622e4d84e28e7d1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:16.688534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d396624-097c-48a3-bcca-55df8a35be87 None admin] RBAC: Authorizing `identity:create_grant(project_id=40a919ef01e24408a577241567900855, user_id=0ddd4ee440f94d37b06ac5253f07c951, role_id=bde14daa0b464f6f9622e4d84e28e7d1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:16.689998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d396624-097c-48a3-bcca-55df8a35be87 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:16.708108 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 567/1123] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:24:16 2022] PUT /identity/v3/projects/40a919ef01e24408a577241567900855/users/0ddd4ee440f94d37b06ac5253f07c951/roles/bde14daa0b464f6f9622e4d84e28e7d1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:18.701169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5004c6-c086-4007-b00c-2074eca49965 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:18.702122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5004c6-c086-4007-b00c-2074eca49965 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:18.702414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5004c6-c086-4007-b00c-2074eca49965 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:18.704654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 557/1124] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:18.711248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc881630-b66d-4359-a20e-c5bcb34716e5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:18.711671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc881630-b66d-4359-a20e-c5bcb34716e5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:18.711777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc881630-b66d-4359-a20e-c5bcb34716e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:18.719455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:18.734306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fc881630-b66d-4359-a20e-c5bcb34716e5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:18.735520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fc881630-b66d-4359-a20e-c5bcb34716e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:18.806320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 568/1125] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:18.832799 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5004c6-c086-4007-b00c-2074eca49965 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:18.885772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:18.942696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5004c6-c086-4007-b00c-2074eca49965 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:18.977897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca217b1e-b6ae-4415-9525-608ea857eb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:18.978675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca217b1e-b6ae-4415-9525-608ea857eb1e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:18.978947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca217b1e-b6ae-4415-9525-608ea857eb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:18.979214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca217b1e-b6ae-4415-9525-608ea857eb1e None admin] PATH_INFO: `/v3/roles/bde14daa0b464f6f9622e4d84e28e7d1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:18.980707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca217b1e-b6ae-4415-9525-608ea857eb1e None admin] RBAC: Authorizing `identity:get_role(role_id=bde14daa0b464f6f9622e4d84e28e7d1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:18.982097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca217b1e-b6ae-4415-9525-608ea857eb1e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:18.983630 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 558/1126] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:18 2022] GET /identity/v3/roles/bde14daa0b464f6f9622e4d84e28e7d1 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:18.990176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc881630-b66d-4359-a20e-c5bcb34716e5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:19.102811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc881630-b66d-4359-a20e-c5bcb34716e5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:19.132700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:19.138688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233d678-382c-4a94-920e-1aeff9b53927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:19.139608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233d678-382c-4a94-920e-1aeff9b53927 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:19.139886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233d678-382c-4a94-920e-1aeff9b53927 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:19.140144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233d678-382c-4a94-920e-1aeff9b53927 None admin] PATH_INFO: `/v3/users/0ddd4ee440f94d37b06ac5253f07c951` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:19.141389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233d678-382c-4a94-920e-1aeff9b53927 None admin] RBAC: Authorizing `identity:get_user(user_id=0ddd4ee440f94d37b06ac5253f07c951)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:19.142820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233d678-382c-4a94-920e-1aeff9b53927 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:19.144448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 569/1127] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:18 2022] GET /identity/v3/users/0ddd4ee440f94d37b06ac5253f07c951 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:19.150462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca217b1e-b6ae-4415-9525-608ea857eb1e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:19.259902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca217b1e-b6ae-4415-9525-608ea857eb1e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:19.273080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:19.296244 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8ed1d-ccdc-439a-97a6-359715c60bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:19.296930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8ed1d-ccdc-439a-97a6-359715c60bba None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:19.297208 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8ed1d-ccdc-439a-97a6-359715c60bba None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:19.297478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8ed1d-ccdc-439a-97a6-359715c60bba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:19.365851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8ed1d-ccdc-439a-97a6-359715c60bba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:19.367280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8ed1d-ccdc-439a-97a6-359715c60bba None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:19.471120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 559/1128] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:24:19.477310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233d678-382c-4a94-920e-1aeff9b53927 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:19.580623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233d678-382c-4a94-920e-1aeff9b53927 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:19.627987 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956c0cc-8da3-42e5-8fbb-de5d61097b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:19.628868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956c0cc-8da3-42e5-8fbb-de5d61097b41 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:19.629152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956c0cc-8da3-42e5-8fbb-de5d61097b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:19.629426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956c0cc-8da3-42e5-8fbb-de5d61097b41 None admin] PATH_INFO: `/v3/projects/40a919ef01e24408a577241567900855` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:19.630694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956c0cc-8da3-42e5-8fbb-de5d61097b41 None admin] RBAC: Authorizing `identity:get_project(project_id=40a919ef01e24408a577241567900855)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:19.632122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956c0cc-8da3-42e5-8fbb-de5d61097b41 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:19.633825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 570/1129] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:19 2022] GET /identity/v3/projects/40a919ef01e24408a577241567900855 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:19.643515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8ed1d-ccdc-439a-97a6-359715c60bba None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:19.662810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:19.765653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8ed1d-ccdc-439a-97a6-359715c60bba None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:19.801469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f002b-d829-4f68-ae79-614e38b1be9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:19.802114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f002b-d829-4f68-ae79-614e38b1be9f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:19.802400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f002b-d829-4f68-ae79-614e38b1be9f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:19.802676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f002b-d829-4f68-ae79-614e38b1be9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:19.803507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f002b-d829-4f68-ae79-614e38b1be9f None admin] RBAC: Adding query filter params (role.id=bde14daa0b464f6f9622e4d84e28e7d1, scope.project.id=40a919ef01e24408a577241567900855, user.id=0ddd4ee440f94d37b06ac5253f07c951) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:19.803822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f002b-d829-4f68-ae79-614e38b1be9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:19.805282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f002b-d829-4f68-ae79-614e38b1be9f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:19.811665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 560/1130] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:24:19 2022] GET /identity/v3/role_assignments?user.id=0ddd4ee440f94d37b06ac5253f07c951&scope.project.id=40a919ef01e24408a577241567900855&role.id=bde14daa0b464f6f9622e4d84e28e7d1 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:21.812596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d505f9-6df9-4e1a-bce2-7b1879e1470e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:21.813176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d505f9-6df9-4e1a-bce2-7b1879e1470e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:21.813176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d505f9-6df9-4e1a-bce2-7b1879e1470e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:21.814473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 571/1131] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:21.820790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3f608-539b-4eb1-89fb-2da0c069c56c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:21.821480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3f608-539b-4eb1-89fb-2da0c069c56c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:21.821694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3f608-539b-4eb1-89fb-2da0c069c56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:21.847442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-76c3f608-539b-4eb1-89fb-2da0c069c56c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:21.848614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-76c3f608-539b-4eb1-89fb-2da0c069c56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:21.921642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 561/1132] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:21.950522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d505f9-6df9-4e1a-bce2-7b1879e1470e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:22.023392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:22.067929 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d505f9-6df9-4e1a-bce2-7b1879e1470e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:22.104542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ce93e-2498-48d6-a72e-1765d47e7f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:22.105362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ce93e-2498-48d6-a72e-1765d47e7f5a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:22.105646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ce93e-2498-48d6-a72e-1765d47e7f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:22.105922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ce93e-2498-48d6-a72e-1765d47e7f5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:22.107129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ce93e-2498-48d6-a72e-1765d47e7f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:22.108616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ce93e-2498-48d6-a72e-1765d47e7f5a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:22.110246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 572/1133] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:24:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:22.116279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3f608-539b-4eb1-89fb-2da0c069c56c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:22.124662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:22.223391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3f608-539b-4eb1-89fb-2da0c069c56c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:22.261614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f24cf5-ecdf-4469-8883-64e733eeec5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:22.262255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f24cf5-ecdf-4469-8883-64e733eeec5b None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:22.262530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f24cf5-ecdf-4469-8883-64e733eeec5b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:22.262798 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f24cf5-ecdf-4469-8883-64e733eeec5b None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:22.263495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f24cf5-ecdf-4469-8883-64e733eeec5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:22.265607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f24cf5-ecdf-4469-8883-64e733eeec5b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:22.298092 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 562/1134] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:24:22 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:24:24.319076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133f9d0-7df4-4547-9faf-05746ff64198 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:24.319972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133f9d0-7df4-4547-9faf-05746ff64198 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:24.319972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133f9d0-7df4-4547-9faf-05746ff64198 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:24.322214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 573/1135] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:24.329271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a71ed-ce2e-4e15-9359-1d74ca1edfa4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:24.329661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a71ed-ce2e-4e15-9359-1d74ca1edfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:24.329832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a71ed-ce2e-4e15-9359-1d74ca1edfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:24.349766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0c0a71ed-ce2e-4e15-9359-1d74ca1edfa4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:24.350974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0c0a71ed-ce2e-4e15-9359-1d74ca1edfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:24.409753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:24.424444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 563/1136] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:24.449356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133f9d0-7df4-4547-9faf-05746ff64198 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:24.567576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133f9d0-7df4-4547-9faf-05746ff64198 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:24.610093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e57c57-ce77-4bbb-9836-8f6cabc5e891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:24.610813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e57c57-ce77-4bbb-9836-8f6cabc5e891 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:24.611099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e57c57-ce77-4bbb-9836-8f6cabc5e891 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:24.611369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e57c57-ce77-4bbb-9836-8f6cabc5e891 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:24.612663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e57c57-ce77-4bbb-9836-8f6cabc5e891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:24.614134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e57c57-ce77-4bbb-9836-8f6cabc5e891 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:24.615752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 574/1137] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:24:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:24.621709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a71ed-ce2e-4e15-9359-1d74ca1edfa4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:24.733312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a71ed-ce2e-4e15-9359-1d74ca1edfa4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:24.769905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16392aa4-1c1f-46f2-a4b2-1e57995bf9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:24.770570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16392aa4-1c1f-46f2-a4b2-1e57995bf9d9 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:24.770849 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16392aa4-1c1f-46f2-a4b2-1e57995bf9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:24.771123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16392aa4-1c1f-46f2-a4b2-1e57995bf9d9 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:24.771840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16392aa4-1c1f-46f2-a4b2-1e57995bf9d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:24.773310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16392aa4-1c1f-46f2-a4b2-1e57995bf9d9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:24.798225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 564/1138] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:24:24 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:24:26.825214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b2bb8-550f-42c5-81bc-8090b1fed0e3 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:26.826017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b2bb8-550f-42c5-81bc-8090b1fed0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:26.826017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b2bb8-550f-42c5-81bc-8090b1fed0e3 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:26.828156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 575/1139] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:26.836434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76077215-a0dc-487a-a1aa-a03d343421b7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:26.837179 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76077215-a0dc-487a-a1aa-a03d343421b7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:26.837179 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76077215-a0dc-487a-a1aa-a03d343421b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:26.855247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-76077215-a0dc-487a-a1aa-a03d343421b7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:26.856420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-76077215-a0dc-487a-a1aa-a03d343421b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:26.899393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:26.931794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 565/1140] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:24:26.955562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b2bb8-550f-42c5-81bc-8090b1fed0e3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:26.959955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:27.088280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b2bb8-550f-42c5-81bc-8090b1fed0e3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:27.125325 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bbcaf-a82d-41ed-ad50-15fe9cac060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:27.126106 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bbcaf-a82d-41ed-ad50-15fe9cac060f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:27.126385 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bbcaf-a82d-41ed-ad50-15fe9cac060f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:27.126659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bbcaf-a82d-41ed-ad50-15fe9cac060f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:27.137958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bbcaf-a82d-41ed-ad50-15fe9cac060f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f02bbcaf-a82d-41ed-ad50-15fe9cac060f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:27.145642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:27.146736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:27.147801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 576/1141] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:27.153435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76077215-a0dc-487a-a1aa-a03d343421b7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:27.268332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76077215-a0dc-487a-a1aa-a03d343421b7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:27.303927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aeffe4-5c2a-4114-9c89-9b40726efe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:27.304579 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aeffe4-5c2a-4114-9c89-9b40726efe88 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:27.304897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aeffe4-5c2a-4114-9c89-9b40726efe88 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:27.305172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aeffe4-5c2a-4114-9c89-9b40726efe88 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:27.305922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aeffe4-5c2a-4114-9c89-9b40726efe88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:27.306225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aeffe4-5c2a-4114-9c89-9b40726efe88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:27.307596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aeffe4-5c2a-4114-9c89-9b40726efe88 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:27.315942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 566/1142] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:24:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:27.324680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bbcaf-a82d-41ed-ad50-15fe9cac060f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:27.391163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:27.432055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bbcaf-a82d-41ed-ad50-15fe9cac060f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:27.466207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b3458-29a8-4d14-8dd8-e98d9d0e356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:27.467170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b3458-29a8-4d14-8dd8-e98d9d0e356f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:27.467441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b3458-29a8-4d14-8dd8-e98d9d0e356f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:27.467716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b3458-29a8-4d14-8dd8-e98d9d0e356f None admin] PATH_INFO: `/v3/users/40f3991538f34034a785aea93c0779c7` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:27.481078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b3458-29a8-4d14-8dd8-e98d9d0e356f None admin] RBAC: Authorizing `identity:get_user(user_id=40f3991538f34034a785aea93c0779c7)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:27.482577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b3458-29a8-4d14-8dd8-e98d9d0e356f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:27.484213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 577/1143] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:27 2022] GET /identity/v3/users/40f3991538f34034a785aea93c0779c7 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:27.490386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aeffe4-5c2a-4114-9c89-9b40726efe88 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:27.530091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:27.596615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aeffe4-5c2a-4114-9c89-9b40726efe88 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:27.631420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4251-652e-4506-a7a6-0b94dda5146b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:27.632061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4251-652e-4506-a7a6-0b94dda5146b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:27.632339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4251-652e-4506-a7a6-0b94dda5146b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:27.632612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4251-652e-4506-a7a6-0b94dda5146b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:27.704711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e4251-652e-4506-a7a6-0b94dda5146b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:27.706231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e4251-652e-4506-a7a6-0b94dda5146b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:27.741055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:27.820994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 567/1144] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:24:27.827176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b3458-29a8-4d14-8dd8-e98d9d0e356f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:27.937678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b3458-29a8-4d14-8dd8-e98d9d0e356f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:27.961200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:27.973885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6f47c-8260-4aaf-b029-bc351defd5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:27.974733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6f47c-8260-4aaf-b029-bc351defd5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:27.975010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6f47c-8260-4aaf-b029-bc351defd5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:27.975301 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6f47c-8260-4aaf-b029-bc351defd5a2 None admin] PATH_INFO: `/v3/projects/c417b3f6584f42f3b83e6df23ea3d082` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:27.976456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6f47c-8260-4aaf-b029-bc351defd5a2 None admin] RBAC: Authorizing `identity:get_project(project_id=c417b3f6584f42f3b83e6df23ea3d082)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:27.977935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6f47c-8260-4aaf-b029-bc351defd5a2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:27.979703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 578/1145] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:27 2022] GET /identity/v3/projects/c417b3f6584f42f3b83e6df23ea3d082 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:27.986965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4251-652e-4506-a7a6-0b94dda5146b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:28.089369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4251-652e-4506-a7a6-0b94dda5146b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:28.124938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090d1d9-d391-4936-942e-f41843ebcd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:28.125571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090d1d9-d391-4936-942e-f41843ebcd02 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:28.125845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090d1d9-d391-4936-942e-f41843ebcd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:28.126117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090d1d9-d391-4936-942e-f41843ebcd02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:28.126942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090d1d9-d391-4936-942e-f41843ebcd02 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=c417b3f6584f42f3b83e6df23ea3d082, user.id=40f3991538f34034a785aea93c0779c7) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:28.127245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090d1d9-d391-4936-942e-f41843ebcd02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:28.128659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090d1d9-d391-4936-942e-f41843ebcd02 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:28.131545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:28.134809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 568/1146] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:24:27 2022] GET /identity/v3/role_assignments?user.id=40f3991538f34034a785aea93c0779c7&scope.project.id=c417b3f6584f42f3b83e6df23ea3d082&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:24:30.105503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a8afe-c9c9-487b-8d1d-84d6a043bb29 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:30.106081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a8afe-c9c9-487b-8d1d-84d6a043bb29 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:30.106081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a8afe-c9c9-487b-8d1d-84d6a043bb29 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:30.107456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 579/1147] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:30.113816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1621bf-ae68-4551-8b4a-e6bf33b383b3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:30.114573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1621bf-ae68-4551-8b4a-e6bf33b383b3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:30.114724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1621bf-ae68-4551-8b4a-e6bf33b383b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:30.136073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1c1621bf-ae68-4551-8b4a-e6bf33b383b3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:30.137291 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1c1621bf-ae68-4551-8b4a-e6bf33b383b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:30.213480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 569/1148] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:30.237246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a8afe-c9c9-487b-8d1d-84d6a043bb29 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:30.367031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a8afe-c9c9-487b-8d1d-84d6a043bb29 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:30.410668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112d705-230d-47e7-9b00-e5625e14a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:30.411452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112d705-230d-47e7-9b00-e5625e14a055 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:30.411731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112d705-230d-47e7-9b00-e5625e14a055 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:30.412007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112d705-230d-47e7-9b00-e5625e14a055 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:30.426112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112d705-230d-47e7-9b00-e5625e14a055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3112d705-230d-47e7-9b00-e5625e14a055 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:30.433720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:30.434761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:30.435733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:30.436914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 580/1149] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:24:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:30.442363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1621bf-ae68-4551-8b4a-e6bf33b383b3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:30.547685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1621bf-ae68-4551-8b4a-e6bf33b383b3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:30.578468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:30.586949 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8368-3992-48ed-9f28-9593e9c54467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:30.587947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8368-3992-48ed-9f28-9593e9c54467 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:30.588370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8368-3992-48ed-9f28-9593e9c54467 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:30.588785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8368-3992-48ed-9f28-9593e9c54467 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:30.590003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a8368-3992-48ed-9f28-9593e9c54467 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:30.590466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a8368-3992-48ed-9f28-9593e9c54467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:30.592554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a8368-3992-48ed-9f28-9593e9c54467 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:30.601194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 570/1150] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:24:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:30.607217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112d705-230d-47e7-9b00-e5625e14a055 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:30.658718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:30.713784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112d705-230d-47e7-9b00-e5625e14a055 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:30.751386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657411b-32b8-4f9e-b3b7-36d3fcea678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:30.752244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657411b-32b8-4f9e-b3b7-36d3fcea678a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:30.752521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657411b-32b8-4f9e-b3b7-36d3fcea678a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:30.752780 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657411b-32b8-4f9e-b3b7-36d3fcea678a None admin] PATH_INFO: `/v3/users/40f3991538f34034a785aea93c0779c7` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:30.754097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657411b-32b8-4f9e-b3b7-36d3fcea678a None admin] RBAC: Authorizing `identity:get_user(user_id=40f3991538f34034a785aea93c0779c7)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:30.755525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657411b-32b8-4f9e-b3b7-36d3fcea678a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:30.757185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 581/1151] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:30 2022] GET /identity/v3/users/40f3991538f34034a785aea93c0779c7 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:30.763435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8368-3992-48ed-9f28-9593e9c54467 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:30.871531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8368-3992-48ed-9f28-9593e9c54467 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:30.907452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14ea22-8d0e-4e2b-8d43-2c8253ed227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:30.908310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14ea22-8d0e-4e2b-8d43-2c8253ed227e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:30.908593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14ea22-8d0e-4e2b-8d43-2c8253ed227e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:30.908966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14ea22-8d0e-4e2b-8d43-2c8253ed227e None admin] PATH_INFO: `/v3/projects/c417b3f6584f42f3b83e6df23ea3d082` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:30.910227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14ea22-8d0e-4e2b-8d43-2c8253ed227e None admin] RBAC: Authorizing `identity:get_project(project_id=c417b3f6584f42f3b83e6df23ea3d082)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:30.911662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14ea22-8d0e-4e2b-8d43-2c8253ed227e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:30.913407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 571/1152] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:30 2022] GET /identity/v3/projects/c417b3f6584f42f3b83e6df23ea3d082 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:30.922666 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657411b-32b8-4f9e-b3b7-36d3fcea678a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:31.035876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657411b-32b8-4f9e-b3b7-36d3fcea678a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:31.055285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:31.073771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35ce97-404e-4db9-b06f-5cc6993f059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:31.074644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35ce97-404e-4db9-b06f-5cc6993f059c None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:31.074919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35ce97-404e-4db9-b06f-5cc6993f059c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:31.075193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35ce97-404e-4db9-b06f-5cc6993f059c None admin] PATH_INFO: `/v3/projects/c417b3f6584f42f3b83e6df23ea3d082/users/40f3991538f34034a785aea93c0779c7/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:31.077450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35ce97-404e-4db9-b06f-5cc6993f059c None admin] RBAC: Authorizing `identity:create_grant(project_id=c417b3f6584f42f3b83e6df23ea3d082, user_id=40f3991538f34034a785aea93c0779c7, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:31.078878 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35ce97-404e-4db9-b06f-5cc6993f059c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:31.094451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 582/1153] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:24:30 2022] PUT /identity/v3/projects/c417b3f6584f42f3b83e6df23ea3d082/users/40f3991538f34034a785aea93c0779c7/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:33.067071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05695a-9c50-48be-9d40-918283b6a8d9 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:33.067634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05695a-9c50-48be-9d40-918283b6a8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:33.067634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05695a-9c50-48be-9d40-918283b6a8d9 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:33.069795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 572/1154] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:33.078542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59480809-f689-406e-a3c6-f459b28642f8 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:33.078947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59480809-f689-406e-a3c6-f459b28642f8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:33.079180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59480809-f689-406e-a3c6-f459b28642f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:33.102269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-59480809-f689-406e-a3c6-f459b28642f8 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:33.103461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-59480809-f689-406e-a3c6-f459b28642f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:33.172014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 583/1155] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:33.196019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05695a-9c50-48be-9d40-918283b6a8d9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:33.262275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:33.305304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05695a-9c50-48be-9d40-918283b6a8d9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:33.340085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafb1ed-2f3b-44ff-86c7-d1b583116ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:33.340781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafb1ed-2f3b-44ff-86c7-d1b583116ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:33.341088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafb1ed-2f3b-44ff-86c7-d1b583116ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:33.341355 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafb1ed-2f3b-44ff-86c7-d1b583116ee8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:33.352440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafb1ed-2f3b-44ff-86c7-d1b583116ee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1bafb1ed-2f3b-44ff-86c7-d1b583116ee8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:33.360126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:33.361173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:24:33.362112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 573/1156] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:24:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:33.367706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59480809-f689-406e-a3c6-f459b28642f8 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:33.485037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59480809-f689-406e-a3c6-f459b28642f8 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:33.494363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:33.520353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03365ad-7f5d-4093-b9bd-5d4e17c44837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:33.521108 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03365ad-7f5d-4093-b9bd-5d4e17c44837 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:33.521392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03365ad-7f5d-4093-b9bd-5d4e17c44837 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:33.521667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03365ad-7f5d-4093-b9bd-5d4e17c44837 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:33.522420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03365ad-7f5d-4093-b9bd-5d4e17c44837 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:33.522725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03365ad-7f5d-4093-b9bd-5d4e17c44837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:33.524103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03365ad-7f5d-4093-b9bd-5d4e17c44837 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:33.532224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 584/1157] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:24:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:33.538316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafb1ed-2f3b-44ff-86c7-d1b583116ee8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:33.655510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafb1ed-2f3b-44ff-86c7-d1b583116ee8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:33.675631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:33.692608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0be8f-d745-4499-81fe-38dede3ae35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:33.693502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0be8f-d745-4499-81fe-38dede3ae35e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:33.693786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0be8f-d745-4499-81fe-38dede3ae35e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:33.694064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0be8f-d745-4499-81fe-38dede3ae35e None admin] PATH_INFO: `/v3/users/40f3991538f34034a785aea93c0779c7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:33.695294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0be8f-d745-4499-81fe-38dede3ae35e None admin] RBAC: Authorizing `identity:get_user(user_id=40f3991538f34034a785aea93c0779c7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:33.696755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0be8f-d745-4499-81fe-38dede3ae35e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:33.698396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 574/1158] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:33 2022] GET /identity/v3/users/40f3991538f34034a785aea93c0779c7 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:33.704224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03365ad-7f5d-4093-b9bd-5d4e17c44837 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:33.806907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03365ad-7f5d-4093-b9bd-5d4e17c44837 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:33.842322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a777fd-7e39-4afc-b078-00aafcf794e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:33.842967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a777fd-7e39-4afc-b078-00aafcf794e1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:33.843248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a777fd-7e39-4afc-b078-00aafcf794e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:33.843526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a777fd-7e39-4afc-b078-00aafcf794e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:33.882852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:33.912355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a777fd-7e39-4afc-b078-00aafcf794e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:33.913752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a777fd-7e39-4afc-b078-00aafcf794e1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:34.017629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 585/1159] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:24:34.024348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0be8f-d745-4499-81fe-38dede3ae35e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:34.136189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0be8f-d745-4499-81fe-38dede3ae35e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:34.171211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae5219-a674-45a6-9d71-7ec7617a0542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:34.171995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae5219-a674-45a6-9d71-7ec7617a0542 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:34.172271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae5219-a674-45a6-9d71-7ec7617a0542 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:34.172539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae5219-a674-45a6-9d71-7ec7617a0542 None admin] PATH_INFO: `/v3/projects/c417b3f6584f42f3b83e6df23ea3d082` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:34.173851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae5219-a674-45a6-9d71-7ec7617a0542 None admin] RBAC: Authorizing `identity:get_project(project_id=c417b3f6584f42f3b83e6df23ea3d082)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:34.175249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae5219-a674-45a6-9d71-7ec7617a0542 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:34.176969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 575/1160] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:34 2022] GET /identity/v3/projects/c417b3f6584f42f3b83e6df23ea3d082 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:34.183825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a777fd-7e39-4afc-b078-00aafcf794e1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:34.262568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:34.302171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a777fd-7e39-4afc-b078-00aafcf794e1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:34.339224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28228ef-fc86-4ac1-ac12-84f740417a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:34.340097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28228ef-fc86-4ac1-ac12-84f740417a9f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:34.340451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28228ef-fc86-4ac1-ac12-84f740417a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:34.340758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28228ef-fc86-4ac1-ac12-84f740417a9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:34.341634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28228ef-fc86-4ac1-ac12-84f740417a9f None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=c417b3f6584f42f3b83e6df23ea3d082, user.id=40f3991538f34034a785aea93c0779c7) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:34.342013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28228ef-fc86-4ac1-ac12-84f740417a9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:34.343622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28228ef-fc86-4ac1-ac12-84f740417a9f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:34.349821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 586/1161] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:24:34 2022] GET /identity/v3/role_assignments?user.id=40f3991538f34034a785aea93c0779c7&scope.project.id=c417b3f6584f42f3b83e6df23ea3d082&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:36.368294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f10b7-b202-4ddc-b760-57959bde7a3c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:36.369126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f10b7-b202-4ddc-b760-57959bde7a3c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:36.369126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f10b7-b202-4ddc-b760-57959bde7a3c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:36.370255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 576/1162] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:36.376581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0505-ac03-46b6-9a8f-776d2ac27028 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:36.377052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0505-ac03-46b6-9a8f-776d2ac27028 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:36.377193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0505-ac03-46b6-9a8f-776d2ac27028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:36.398080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b4ae0505-ac03-46b6-9a8f-776d2ac27028 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:36.399271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b4ae0505-ac03-46b6-9a8f-776d2ac27028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:36.471699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 587/1163] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:36.499129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f10b7-b202-4ddc-b760-57959bde7a3c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:36.561986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:36.608000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f10b7-b202-4ddc-b760-57959bde7a3c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:36.646428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7e361-9f65-4248-afa1-62bd705d4463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:36.646996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7e361-9f65-4248-afa1-62bd705d4463 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:36.647278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7e361-9f65-4248-afa1-62bd705d4463 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:36.647551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7e361-9f65-4248-afa1-62bd705d4463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:36.716675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7e361-9f65-4248-afa1-62bd705d4463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:36.718263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7e361-9f65-4248-afa1-62bd705d4463 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:36.777517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:36.828282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 577/1164] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:24:36.838785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae0505-ac03-46b6-9a8f-776d2ac27028 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:36.910402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:36.951144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae0505-ac03-46b6-9a8f-776d2ac27028 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:36.986061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ce30c-7f0d-4865-a611-be6ed6354e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:36.986843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ce30c-7f0d-4865-a611-be6ed6354e5d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:36.987118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ce30c-7f0d-4865-a611-be6ed6354e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:36.987384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ce30c-7f0d-4865-a611-be6ed6354e5d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:36.994240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ce30c-7f0d-4865-a611-be6ed6354e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:36.995676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ce30c-7f0d-4865-a611-be6ed6354e5d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7e9ce30c-7f0d-4865-a611-be6ed6354e5d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:37.009843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:37.010996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:37.011863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 588/1165] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:24:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:37.017437 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7e361-9f65-4248-afa1-62bd705d4463 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:37.122069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7e361-9f65-4248-afa1-62bd705d4463 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:37.159833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc458c-41ea-4c3f-9bc0-e5ad98392d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:37.160544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc458c-41ea-4c3f-9bc0-e5ad98392d1d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:37.160763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc458c-41ea-4c3f-9bc0-e5ad98392d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:37.161073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc458c-41ea-4c3f-9bc0-e5ad98392d1d None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:37.161800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc458c-41ea-4c3f-9bc0-e5ad98392d1d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:37.162109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc458c-41ea-4c3f-9bc0-e5ad98392d1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:37.163472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc458c-41ea-4c3f-9bc0-e5ad98392d1d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:37.171452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 578/1166] 10.222.0.69 () {56 vars in 1115 bytes} [Fri Jun 24 10:24:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:39.152870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c225bc-5de3-425e-a42f-feaf21aa1369 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:39.153426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c225bc-5de3-425e-a42f-feaf21aa1369 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:39.153426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c225bc-5de3-425e-a42f-feaf21aa1369 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:39.154838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 589/1167] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:39.161106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c501a49-1853-4df2-aaf4-d0bf3669e450 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:39.161761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c501a49-1853-4df2-aaf4-d0bf3669e450 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:39.162034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c501a49-1853-4df2-aaf4-d0bf3669e450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:39.188905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9c501a49-1853-4df2-aaf4-d0bf3669e450 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:39.190105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9c501a49-1853-4df2-aaf4-d0bf3669e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:39.205206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:39.263578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 579/1168] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:39.286841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c225bc-5de3-425e-a42f-feaf21aa1369 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:39.394816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c225bc-5de3-425e-a42f-feaf21aa1369 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:39.422206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:39.434408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae3706-1983-4d6f-9afc-34c4e833d805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:39.435063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae3706-1983-4d6f-9afc-34c4e833d805 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:39.435382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae3706-1983-4d6f-9afc-34c4e833d805 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:39.435766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae3706-1983-4d6f-9afc-34c4e833d805 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:39.436517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae3706-1983-4d6f-9afc-34c4e833d805 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:39.437949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae3706-1983-4d6f-9afc-34c4e833d805 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:39.457699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 590/1169] 10.222.0.69 () {60 vars in 1138 bytes} [Fri Jun 24 10:24:39 2022] POST /identity/v3/domains => generated 267 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:24:41.480089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55aa73-bb3d-4353-b9a4-d85c7fdbf752 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:41.480554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55aa73-bb3d-4353-b9a4-d85c7fdbf752 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:41.480666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55aa73-bb3d-4353-b9a4-d85c7fdbf752 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:41.482056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 580/1170] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:41.487874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c77ae0-2231-47c6-905f-3feb318fc9da None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:41.488308 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c77ae0-2231-47c6-905f-3feb318fc9da None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:41.488397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c77ae0-2231-47c6-905f-3feb318fc9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:41.506427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c9c77ae0-2231-47c6-905f-3feb318fc9da None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:41.507611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c9c77ae0-2231-47c6-905f-3feb318fc9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:41.579730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 591/1171] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:41.603451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55aa73-bb3d-4353-b9a4-d85c7fdbf752 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:41.719936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55aa73-bb3d-4353-b9a4-d85c7fdbf752 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:41.756082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247d7e8-51a3-4ab3-b544-19f1eb9b740f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:41.756964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247d7e8-51a3-4ab3-b544-19f1eb9b740f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:41.757247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247d7e8-51a3-4ab3-b544-19f1eb9b740f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:41.757525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247d7e8-51a3-4ab3-b544-19f1eb9b740f None admin] PATH_INFO: `/v3/domains/f51e971fe3e54c82afff2f6ba7cf343d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:41.768624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247d7e8-51a3-4ab3-b544-19f1eb9b740f None admin] RBAC: Authorizing `identity:get_domain(domain_id=f51e971fe3e54c82afff2f6ba7cf343d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:41.770101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247d7e8-51a3-4ab3-b544-19f1eb9b740f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:41.771736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 581/1172] 10.222.0.69 () {56 vars in 1183 bytes} [Fri Jun 24 10:24:41 2022] GET /identity/v3/domains/f51e971fe3e54c82afff2f6ba7cf343d => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:41.781494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c77ae0-2231-47c6-905f-3feb318fc9da None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:41.887235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c77ae0-2231-47c6-905f-3feb318fc9da None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:41.902819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:41.924180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5bb4c-6bb2-4846-85aa-a00d50d990df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:41.924860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5bb4c-6bb2-4846-85aa-a00d50d990df None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:41.925140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5bb4c-6bb2-4846-85aa-a00d50d990df None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:41.925418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5bb4c-6bb2-4846-85aa-a00d50d990df None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:41.926116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5bb4c-6bb2-4846-85aa-a00d50d990df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:41.927547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5bb4c-6bb2-4846-85aa-a00d50d990df None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:41.954213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 592/1173] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:24:41 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:24:44.082623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a1a9b-b4c2-4368-a7d1-2c629f82d6f2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:44.083435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a1a9b-b4c2-4368-a7d1-2c629f82d6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:44.083435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a1a9b-b4c2-4368-a7d1-2c629f82d6f2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:44.085810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 582/1174] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:44.092036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d508a-6aee-4bdf-80c5-344e2eb89329 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:44.092820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d508a-6aee-4bdf-80c5-344e2eb89329 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:44.092820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d508a-6aee-4bdf-80c5-344e2eb89329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:44.110058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3f0d508a-6aee-4bdf-80c5-344e2eb89329 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:44.111255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f0d508a-6aee-4bdf-80c5-344e2eb89329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:44.181979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 593/1175] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:44.222871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a1a9b-b4c2-4368-a7d1-2c629f82d6f2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:44.238220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:44.337886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a1a9b-b4c2-4368-a7d1-2c629f82d6f2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:44.373150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a193e5-eb92-463f-bccb-3bdfeea2c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:44.374009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a193e5-eb92-463f-bccb-3bdfeea2c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:44.374293 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a193e5-eb92-463f-bccb-3bdfeea2c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:44.374566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a193e5-eb92-463f-bccb-3bdfeea2c5d2 None admin] PATH_INFO: `/v3/domains/f51e971fe3e54c82afff2f6ba7cf343d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:44.375784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a193e5-eb92-463f-bccb-3bdfeea2c5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f51e971fe3e54c82afff2f6ba7cf343d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:44.377238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a193e5-eb92-463f-bccb-3bdfeea2c5d2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:44.378872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 583/1176] 10.222.0.69 () {56 vars in 1183 bytes} [Fri Jun 24 10:24:44 2022] GET /identity/v3/domains/f51e971fe3e54c82afff2f6ba7cf343d => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:44.384992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d508a-6aee-4bdf-80c5-344e2eb89329 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:44.494595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:44.501400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d508a-6aee-4bdf-80c5-344e2eb89329 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:44.538668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ccf50-1dd1-4ed6-8ad0-450f643a55c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:44.539331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ccf50-1dd1-4ed6-8ad0-450f643a55c1 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:44.539611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ccf50-1dd1-4ed6-8ad0-450f643a55c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:44.539894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ccf50-1dd1-4ed6-8ad0-450f643a55c1 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:44.540613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ccf50-1dd1-4ed6-8ad0-450f643a55c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:44.542066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ccf50-1dd1-4ed6-8ad0-450f643a55c1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:44.565868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 594/1177] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:24:44 2022] POST /identity/v3/users => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:24:46.606307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f29ab-56b1-4611-b057-acf77747d841 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:46.606875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f29ab-56b1-4611-b057-acf77747d841 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:46.606875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f29ab-56b1-4611-b057-acf77747d841 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:46.608322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 584/1178] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:46.614654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1be2-47dd-4fe0-beb0-258cc76a5dfc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:46.615401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1be2-47dd-4fe0-beb0-258cc76a5dfc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:46.615538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1be2-47dd-4fe0-beb0-258cc76a5dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:46.636626 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c48d1be2-47dd-4fe0-beb0-258cc76a5dfc None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:46.637836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c48d1be2-47dd-4fe0-beb0-258cc76a5dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:46.707981 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 595/1179] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:46.741672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f29ab-56b1-4611-b057-acf77747d841 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:46.831265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:46.860225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f29ab-56b1-4611-b057-acf77747d841 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:46.893915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1637385-06fc-4001-bb7c-0aae2d36af55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:46.894701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1637385-06fc-4001-bb7c-0aae2d36af55 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:46.894980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1637385-06fc-4001-bb7c-0aae2d36af55 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:46.895277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1637385-06fc-4001-bb7c-0aae2d36af55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:46.907165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1637385-06fc-4001-bb7c-0aae2d36af55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b1637385-06fc-4001-bb7c-0aae2d36af55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:46.914711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:46.915763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:24:46.916716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 585/1180] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:24:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:46.922104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d1be2-47dd-4fe0-beb0-258cc76a5dfc None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:47.015791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:47.035308 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d1be2-47dd-4fe0-beb0-258cc76a5dfc None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:47.070131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0a867-07fd-4b82-b8d0-ab7d107b9dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:47.070798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0a867-07fd-4b82-b8d0-ab7d107b9dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:47.071076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0a867-07fd-4b82-b8d0-ab7d107b9dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:47.071357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0a867-07fd-4b82-b8d0-ab7d107b9dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:47.072137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0a867-07fd-4b82-b8d0-ab7d107b9dd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:47.072445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0a867-07fd-4b82-b8d0-ab7d107b9dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:47.073857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0a867-07fd-4b82-b8d0-ab7d107b9dd0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:47.082198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 596/1181] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:24:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:47.091240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1637385-06fc-4001-bb7c-0aae2d36af55 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:47.207908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1637385-06fc-4001-bb7c-0aae2d36af55 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:47.244886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84e1de-a360-4d70-8217-03106ba76a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:47.245745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84e1de-a360-4d70-8217-03106ba76a2c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:47.246033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84e1de-a360-4d70-8217-03106ba76a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:47.246306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84e1de-a360-4d70-8217-03106ba76a2c None admin] PATH_INFO: `/v3/users/e009683a1f0a4eeca3f3d3c8475b678b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:47.254310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:47.261668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84e1de-a360-4d70-8217-03106ba76a2c None admin] RBAC: Authorizing `identity:get_user(user_id=e009683a1f0a4eeca3f3d3c8475b678b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:47.263121 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84e1de-a360-4d70-8217-03106ba76a2c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:47.265328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 586/1182] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:47 2022] GET /identity/v3/users/e009683a1f0a4eeca3f3d3c8475b678b => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:47.274495 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0a867-07fd-4b82-b8d0-ab7d107b9dd0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:47.388936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0a867-07fd-4b82-b8d0-ab7d107b9dd0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:47.422938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40de9b-780e-42ca-b5aa-48143cfbf6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:47.423568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40de9b-780e-42ca-b5aa-48143cfbf6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:47.423850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40de9b-780e-42ca-b5aa-48143cfbf6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:47.424127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40de9b-780e-42ca-b5aa-48143cfbf6a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:47.438691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:47.495176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40de9b-780e-42ca-b5aa-48143cfbf6a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:47.496635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40de9b-780e-42ca-b5aa-48143cfbf6a4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:47.596611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 597/1183] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:24:47.602744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84e1de-a360-4d70-8217-03106ba76a2c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:47.711673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84e1de-a360-4d70-8217-03106ba76a2c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:47.748348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd60d8a-e233-45db-888d-716f4550914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:47.749231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd60d8a-e233-45db-888d-716f4550914e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:47.749509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd60d8a-e233-45db-888d-716f4550914e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:47.749786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd60d8a-e233-45db-888d-716f4550914e None admin] PATH_INFO: `/v3/projects/ca99a4858b2747278717413737f4a293` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:47.751085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd60d8a-e233-45db-888d-716f4550914e None admin] RBAC: Authorizing `identity:get_project(project_id=ca99a4858b2747278717413737f4a293)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:47.752513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd60d8a-e233-45db-888d-716f4550914e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:47.754247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 587/1184] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:47 2022] GET /identity/v3/projects/ca99a4858b2747278717413737f4a293 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:47.760729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40de9b-780e-42ca-b5aa-48143cfbf6a4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:47.805977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:47.861186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40de9b-780e-42ca-b5aa-48143cfbf6a4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:47.904999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1dd499-38ee-4b4d-863c-3630d9eb54f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:47.905655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dd499-38ee-4b4d-863c-3630d9eb54f9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:47.905939 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dd499-38ee-4b4d-863c-3630d9eb54f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:47.906214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dd499-38ee-4b4d-863c-3630d9eb54f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:47.907056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1dd499-38ee-4b4d-863c-3630d9eb54f9 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=ca99a4858b2747278717413737f4a293, user.id=e009683a1f0a4eeca3f3d3c8475b678b) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:47.907374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1dd499-38ee-4b4d-863c-3630d9eb54f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:47.908838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1dd499-38ee-4b4d-863c-3630d9eb54f9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:47.914799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 598/1185] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:24:47 2022] GET /identity/v3/role_assignments?user.id=e009683a1f0a4eeca3f3d3c8475b678b&scope.project.id=ca99a4858b2747278717413737f4a293&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:49.920458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac413f-1109-4f66-9113-bdd424707b12 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:49.921366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac413f-1109-4f66-9113-bdd424707b12 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:49.921366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac413f-1109-4f66-9113-bdd424707b12 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:49.923606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 588/1186] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:49.929071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd8ff0-eb39-4b7c-8ab7-ebdd5685d02c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:49.929502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd8ff0-eb39-4b7c-8ab7-ebdd5685d02c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:49.929596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd8ff0-eb39-4b7c-8ab7-ebdd5685d02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:49.948770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1cfd8ff0-eb39-4b7c-8ab7-ebdd5685d02c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:49.949982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1cfd8ff0-eb39-4b7c-8ab7-ebdd5685d02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:50.019703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 599/1187] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:50.043300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac413f-1109-4f66-9113-bdd424707b12 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:50.116299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:50.159128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac413f-1109-4f66-9113-bdd424707b12 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:50.194472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45d3a1-1fe0-4e67-9927-20448cc2d6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:50.195182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45d3a1-1fe0-4e67-9927-20448cc2d6de None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:50.195462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45d3a1-1fe0-4e67-9927-20448cc2d6de None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:50.195729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45d3a1-1fe0-4e67-9927-20448cc2d6de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:50.213162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45d3a1-1fe0-4e67-9927-20448cc2d6de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-cd45d3a1-1fe0-4e67-9927-20448cc2d6de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:50.222388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:50.223499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:24:50.224755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 589/1188] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:24:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:24:50.229769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd8ff0-eb39-4b7c-8ab7-ebdd5685d02c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:50.270458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:50.345757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd8ff0-eb39-4b7c-8ab7-ebdd5685d02c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:50.380516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac27c8-965e-44c2-bb95-d667c5c55388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:50.381259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac27c8-965e-44c2-bb95-d667c5c55388 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:50.381533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac27c8-965e-44c2-bb95-d667c5c55388 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:50.381814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac27c8-965e-44c2-bb95-d667c5c55388 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:50.382547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac27c8-965e-44c2-bb95-d667c5c55388 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:50.382853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac27c8-965e-44c2-bb95-d667c5c55388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:50.384219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac27c8-965e-44c2-bb95-d667c5c55388 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:50.392924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 600/1189] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:50.399585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45d3a1-1fe0-4e67-9927-20448cc2d6de None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:50.512359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45d3a1-1fe0-4e67-9927-20448cc2d6de None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:50.535992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:50.547914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272be9e-78fc-4f49-95fa-6be948a438b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:50.548766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272be9e-78fc-4f49-95fa-6be948a438b0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:50.549097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272be9e-78fc-4f49-95fa-6be948a438b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:50.549361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272be9e-78fc-4f49-95fa-6be948a438b0 None admin] PATH_INFO: `/v3/users/e009683a1f0a4eeca3f3d3c8475b678b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:50.550576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272be9e-78fc-4f49-95fa-6be948a438b0 None admin] RBAC: Authorizing `identity:get_user(user_id=e009683a1f0a4eeca3f3d3c8475b678b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:50.551994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272be9e-78fc-4f49-95fa-6be948a438b0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:50.553616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 590/1190] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:50 2022] GET /identity/v3/users/e009683a1f0a4eeca3f3d3c8475b678b => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:50.562320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac27c8-965e-44c2-bb95-d667c5c55388 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:50.661953 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:50.666638 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac27c8-965e-44c2-bb95-d667c5c55388 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:50.704610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659f9a7-7dce-494c-b0b8-976faa1b2249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:50.705550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659f9a7-7dce-494c-b0b8-976faa1b2249 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:50.705830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659f9a7-7dce-494c-b0b8-976faa1b2249 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:50.706106 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659f9a7-7dce-494c-b0b8-976faa1b2249 None admin] PATH_INFO: `/v3/projects/ca99a4858b2747278717413737f4a293` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:50.707836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659f9a7-7dce-494c-b0b8-976faa1b2249 None admin] RBAC: Authorizing `identity:get_project(project_id=ca99a4858b2747278717413737f4a293)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:50.709276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659f9a7-7dce-494c-b0b8-976faa1b2249 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:50.710971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 601/1191] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:50 2022] GET /identity/v3/projects/ca99a4858b2747278717413737f4a293 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:50.717393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272be9e-78fc-4f49-95fa-6be948a438b0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:50.827718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272be9e-78fc-4f49-95fa-6be948a438b0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:50.864678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffae2b0-a787-4332-b2dc-b6d5c6a73228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:50.865590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae2b0-a787-4332-b2dc-b6d5c6a73228 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:50.865871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae2b0-a787-4332-b2dc-b6d5c6a73228 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:50.866143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae2b0-a787-4332-b2dc-b6d5c6a73228 None admin] PATH_INFO: `/v3/projects/ca99a4858b2747278717413737f4a293/users/e009683a1f0a4eeca3f3d3c8475b678b/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:50.868352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffae2b0-a787-4332-b2dc-b6d5c6a73228 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca99a4858b2747278717413737f4a293, user_id=e009683a1f0a4eeca3f3d3c8475b678b, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:50.869809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffae2b0-a787-4332-b2dc-b6d5c6a73228 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:50.886697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 591/1192] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:24:50 2022] PUT /identity/v3/projects/ca99a4858b2747278717413737f4a293/users/e009683a1f0a4eeca3f3d3c8475b678b/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:52.883070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c281a-0f39-4bc9-ac1a-2252b8a1dc79 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:52.883618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c281a-0f39-4bc9-ac1a-2252b8a1dc79 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:52.883618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c281a-0f39-4bc9-ac1a-2252b8a1dc79 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:52.884961 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 602/1193] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:24:52.891395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151165a-995e-49a4-8c3e-2ff09f59476f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:52.892149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151165a-995e-49a4-8c3e-2ff09f59476f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:52.892300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151165a-995e-49a4-8c3e-2ff09f59476f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:52.921324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1151165a-995e-49a4-8c3e-2ff09f59476f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:52.922522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1151165a-995e-49a4-8c3e-2ff09f59476f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:52.972515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:53.000248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 592/1194] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:24:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:24:53.025210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c281a-0f39-4bc9-ac1a-2252b8a1dc79 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:53.137659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c281a-0f39-4bc9-ac1a-2252b8a1dc79 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:53.172652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db1a51-6340-4496-a9e5-61829ddd5487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:53.173462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db1a51-6340-4496-a9e5-61829ddd5487 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:53.173744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db1a51-6340-4496-a9e5-61829ddd5487 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:53.174021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db1a51-6340-4496-a9e5-61829ddd5487 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:53.185675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db1a51-6340-4496-a9e5-61829ddd5487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-49db1a51-6340-4496-a9e5-61829ddd5487 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:24:53.193175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:24:53.194228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:24:53.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 603/1195] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:24:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:24:53.203466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151165a-995e-49a4-8c3e-2ff09f59476f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:53.315026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151165a-995e-49a4-8c3e-2ff09f59476f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:53.352026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e198959-b85e-434e-92f6-e885974eda56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:53.352676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e198959-b85e-434e-92f6-e885974eda56 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:53.352989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e198959-b85e-434e-92f6-e885974eda56 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:53.353265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e198959-b85e-434e-92f6-e885974eda56 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:53.354015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e198959-b85e-434e-92f6-e885974eda56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:53.354324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e198959-b85e-434e-92f6-e885974eda56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:53.355685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e198959-b85e-434e-92f6-e885974eda56 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:53.365168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 593/1196] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:24:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:53.371611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db1a51-6340-4496-a9e5-61829ddd5487 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:53.405630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:53.482475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db1a51-6340-4496-a9e5-61829ddd5487 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:53.518382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e6a9f-278f-4b01-ba9c-c670db3f23f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:53.519240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6a9f-278f-4b01-ba9c-c670db3f23f3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:53.519516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6a9f-278f-4b01-ba9c-c670db3f23f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:53.519783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6a9f-278f-4b01-ba9c-c670db3f23f3 None admin] PATH_INFO: `/v3/users/e009683a1f0a4eeca3f3d3c8475b678b` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:53.521104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e6a9f-278f-4b01-ba9c-c670db3f23f3 None admin] RBAC: Authorizing `identity:get_user(user_id=e009683a1f0a4eeca3f3d3c8475b678b)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:53.522534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e6a9f-278f-4b01-ba9c-c670db3f23f3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:53.524163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 604/1197] 10.222.0.69 () {56 vars in 1177 bytes} [Fri Jun 24 10:24:53 2022] GET /identity/v3/users/e009683a1f0a4eeca3f3d3c8475b678b => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:53.534281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e198959-b85e-434e-92f6-e885974eda56 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:53.586296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:53.644170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e198959-b85e-434e-92f6-e885974eda56 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:53.678814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3824-f953-4bd1-92a0-c76202a0c6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:53.679388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3824-f953-4bd1-92a0-c76202a0c6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:53.679646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3824-f953-4bd1-92a0-c76202a0c6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:53.679961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3824-f953-4bd1-92a0-c76202a0c6f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:53.751919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a3824-f953-4bd1-92a0-c76202a0c6f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:53.753389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a3824-f953-4bd1-92a0-c76202a0c6f2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:53.798730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:53.859675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 594/1198] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:24:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:24:53.865950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e6a9f-278f-4b01-ba9c-c670db3f23f3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:53.986355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:53.996204 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e6a9f-278f-4b01-ba9c-c670db3f23f3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:54.035714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca58492-732d-4c78-851f-cc3d9beafdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:54.036561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58492-732d-4c78-851f-cc3d9beafdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:54.036872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58492-732d-4c78-851f-cc3d9beafdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:54.037154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58492-732d-4c78-851f-cc3d9beafdf4 None admin] PATH_INFO: `/v3/projects/ca99a4858b2747278717413737f4a293` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:54.038356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca58492-732d-4c78-851f-cc3d9beafdf4 None admin] RBAC: Authorizing `identity:get_project(project_id=ca99a4858b2747278717413737f4a293)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:54.039787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca58492-732d-4c78-851f-cc3d9beafdf4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:54.041515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 605/1199] 10.222.0.69 () {56 vars in 1186 bytes} [Fri Jun 24 10:24:53 2022] GET /identity/v3/projects/ca99a4858b2747278717413737f4a293 => generated 361 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:54.048641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3824-f953-4bd1-92a0-c76202a0c6f2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:54.164067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3824-f953-4bd1-92a0-c76202a0c6f2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:54.199653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31e4e0-cdc7-43bf-96d9-69b3ed8faa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:54.200291 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31e4e0-cdc7-43bf-96d9-69b3ed8faa64 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:54.200575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31e4e0-cdc7-43bf-96d9-69b3ed8faa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:54.200894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31e4e0-cdc7-43bf-96d9-69b3ed8faa64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:54.201900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31e4e0-cdc7-43bf-96d9-69b3ed8faa64 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=ca99a4858b2747278717413737f4a293, user.id=e009683a1f0a4eeca3f3d3c8475b678b) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:54.202217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31e4e0-cdc7-43bf-96d9-69b3ed8faa64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:54.203651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31e4e0-cdc7-43bf-96d9-69b3ed8faa64 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:54.210021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 595/1200] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:24:54 2022] GET /identity/v3/role_assignments?user.id=e009683a1f0a4eeca3f3d3c8475b678b&scope.project.id=ca99a4858b2747278717413737f4a293&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:04.031128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=18 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:04.207328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:49.336811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a851db-41d8-4042-bd71-5f5f8b508244 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:49.336811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a851db-41d8-4042-bd71-5f5f8b508244 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:49.336811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a851db-41d8-4042-bd71-5f5f8b508244 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:49.338158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 606/1201] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:25:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:25:49.344753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec92033-b1ef-4a92-9f74-b671279af43c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:49.345347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec92033-b1ef-4a92-9f74-b671279af43c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:49.345347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec92033-b1ef-4a92-9f74-b671279af43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:49.365109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fec92033-b1ef-4a92-9f74-b671279af43c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:25:49.366322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fec92033-b1ef-4a92-9f74-b671279af43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:25:49.445204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 596/1202] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:25:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:25:49.471183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a851db-41d8-4042-bd71-5f5f8b508244 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:49.593800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a851db-41d8-4042-bd71-5f5f8b508244 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:49.638281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c79b2-66eb-4358-9f4b-1b17941caa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:49.639189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c79b2-66eb-4358-9f4b-1b17941caa66 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:49.639468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c79b2-66eb-4358-9f4b-1b17941caa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:49.639765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c79b2-66eb-4358-9f4b-1b17941caa66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:49.652785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c79b2-66eb-4358-9f4b-1b17941caa66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:49.653511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c79b2-66eb-4358-9f4b-1b17941caa66 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-154c79b2-66eb-4358-9f4b-1b17941caa66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:25:49.673620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:49.674874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:49.675819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:25:49.675819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:25:49.675819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:49.675819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:25:49.679994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 607/1203] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:49.681435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec92033-b1ef-4a92-9f74-b671279af43c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:49.811340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec92033-b1ef-4a92-9f74-b671279af43c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:49.850784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:49.857940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67d8c6-a4fd-445b-acba-225a372bd223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:49.858771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67d8c6-a4fd-445b-acba-225a372bd223 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:49.859151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67d8c6-a4fd-445b-acba-225a372bd223 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:49.859458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67d8c6-a4fd-445b-acba-225a372bd223 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:49.860439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67d8c6-a4fd-445b-acba-225a372bd223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:49.860762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67d8c6-a4fd-445b-acba-225a372bd223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:49.862727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67d8c6-a4fd-445b-acba-225a372bd223 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:49.877049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 597/1204] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:49.882570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c79b2-66eb-4358-9f4b-1b17941caa66 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:49.953686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:50.015891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c79b2-66eb-4358-9f4b-1b17941caa66 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:50.059519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e72ae-6136-4ed5-997d-f2d8dfda67ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:50.060587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e72ae-6136-4ed5-997d-f2d8dfda67ba None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:50.061014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e72ae-6136-4ed5-997d-f2d8dfda67ba None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:50.061195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e72ae-6136-4ed5-997d-f2d8dfda67ba None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:50.062025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e72ae-6136-4ed5-997d-f2d8dfda67ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:50.063988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e72ae-6136-4ed5-997d-f2d8dfda67ba None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:50.100770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 608/1205] 10.222.0.69 () {60 vars in 1132 bytes} [Fri Jun 24 10:25:49 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:25:52.351826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd76e2-6988-4634-93e3-86c5bd971e4d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:52.352375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd76e2-6988-4634-93e3-86c5bd971e4d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:52.352375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd76e2-6988-4634-93e3-86c5bd971e4d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:52.354004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 598/1206] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:25:52.360331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1f34a-6ff1-4fd5-a9d3-29b5d35bb737 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:52.360874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1f34a-6ff1-4fd5-a9d3-29b5d35bb737 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:52.361110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1f34a-6ff1-4fd5-a9d3-29b5d35bb737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:52.379439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-26c1f34a-6ff1-4fd5-a9d3-29b5d35bb737 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:25:52.380661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-26c1f34a-6ff1-4fd5-a9d3-29b5d35bb737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:25:52.466556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 609/1207] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:25:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:25:52.492887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd76e2-6988-4634-93e3-86c5bd971e4d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:52.617565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd76e2-6988-4634-93e3-86c5bd971e4d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:52.654438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a396337-682f-48c4-b2cf-4cd09c4be2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:52.655248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a396337-682f-48c4-b2cf-4cd09c4be2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:52.655639 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a396337-682f-48c4-b2cf-4cd09c4be2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:52.655992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a396337-682f-48c4-b2cf-4cd09c4be2c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:52.667959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a396337-682f-48c4-b2cf-4cd09c4be2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-9a396337-682f-48c4-b2cf-4cd09c4be2c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:25:52.676269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:52.677514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:25:52.678519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 599/1208] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:25:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:52.685964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1f34a-6ff1-4fd5-a9d3-29b5d35bb737 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:52.737638 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:52.809113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1f34a-6ff1-4fd5-a9d3-29b5d35bb737 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:52.846074 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a7c47-915f-46c5-9761-81b789d03982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:52.846829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7c47-915f-46c5-9761-81b789d03982 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:52.847100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7c47-915f-46c5-9761-81b789d03982 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:52.847348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7c47-915f-46c5-9761-81b789d03982 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:52.848204 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a7c47-915f-46c5-9761-81b789d03982 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:52.848530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a7c47-915f-46c5-9761-81b789d03982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:52.850025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a7c47-915f-46c5-9761-81b789d03982 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:52.864218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 610/1209] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:25:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:52.866270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a396337-682f-48c4-b2cf-4cd09c4be2c7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:52.923645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:52.982576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a396337-682f-48c4-b2cf-4cd09c4be2c7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:53.025790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e644e3-26e6-4e41-81c3-6ae00822dceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:53.026627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e644e3-26e6-4e41-81c3-6ae00822dceb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:53.026940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e644e3-26e6-4e41-81c3-6ae00822dceb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:53.027324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e644e3-26e6-4e41-81c3-6ae00822dceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:53.039497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e644e3-26e6-4e41-81c3-6ae00822dceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:53.041249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e644e3-26e6-4e41-81c3-6ae00822dceb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-86e644e3-26e6-4e41-81c3-6ae00822dceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:25:53.061224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:53.062387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:53.063460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:25:53.063460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:25:53.063460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:53.063460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:25:53.063460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 600/1210] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:53.068875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a7c47-915f-46c5-9761-81b789d03982 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:53.174765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:53.178295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a7c47-915f-46c5-9761-81b789d03982 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:53.221480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b120b-8506-477c-b4cb-0bf484367692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:53.222336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b120b-8506-477c-b4cb-0bf484367692 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:53.222664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b120b-8506-477c-b4cb-0bf484367692 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:53.223083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b120b-8506-477c-b4cb-0bf484367692 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:53.223895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b120b-8506-477c-b4cb-0bf484367692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:53.224231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b120b-8506-477c-b4cb-0bf484367692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:53.225756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b120b-8506-477c-b4cb-0bf484367692 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:53.240169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 611/1211] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:53.245877 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e644e3-26e6-4e41-81c3-6ae00822dceb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:53.368065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e644e3-26e6-4e41-81c3-6ae00822dceb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:53.369879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:53.410607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4c7ca-7a3b-434c-9644-5a5bfde3503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:53.411545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4c7ca-7a3b-434c-9644-5a5bfde3503c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:53.411907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4c7ca-7a3b-434c-9644-5a5bfde3503c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:53.412184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4c7ca-7a3b-434c-9644-5a5bfde3503c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:53.420501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4c7ca-7a3b-434c-9644-5a5bfde3503c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:53.422160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4c7ca-7a3b-434c-9644-5a5bfde3503c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8ed4c7ca-7a3b-434c-9644-5a5bfde3503c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:25:53.440261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:53.441320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:25:53.448803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 601/1212] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:25:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:53.449585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b120b-8506-477c-b4cb-0bf484367692 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:53.562931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b120b-8506-477c-b4cb-0bf484367692 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:53.606033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9c9e1-b9df-44e7-84bc-dc3123a00d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:53.606741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9c9e1-b9df-44e7-84bc-dc3123a00d54 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:53.607033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9c9e1-b9df-44e7-84bc-dc3123a00d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:53.607312 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9c9e1-b9df-44e7-84bc-dc3123a00d54 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:53.608184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9c9e1-b9df-44e7-84bc-dc3123a00d54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:53.608494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9c9e1-b9df-44e7-84bc-dc3123a00d54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:53.610236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9c9e1-b9df-44e7-84bc-dc3123a00d54 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:53.631654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 612/1213] 10.222.0.69 () {56 vars in 1143 bytes} [Fri Jun 24 10:25:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:53.637706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4c7ca-7a3b-434c-9644-5a5bfde3503c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:53.751175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4c7ca-7a3b-434c-9644-5a5bfde3503c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:53.792423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d80bc-a5b3-4520-9983-6889fc7fba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:53.793147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d80bc-a5b3-4520-9983-6889fc7fba37 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:53.793457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d80bc-a5b3-4520-9983-6889fc7fba37 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:53.793766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d80bc-a5b3-4520-9983-6889fc7fba37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:53.826919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:53.874766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d80bc-a5b3-4520-9983-6889fc7fba37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:53.876464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d80bc-a5b3-4520-9983-6889fc7fba37 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:53.995478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 602/1214] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:25:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:25:53.996781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9c9e1-b9df-44e7-84bc-dc3123a00d54 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:54.017541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:54.106985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9c9e1-b9df-44e7-84bc-dc3123a00d54 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:54.145151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1aa69-3d27-4095-bc6a-3ff69f39b8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:54.146234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1aa69-3d27-4095-bc6a-3ff69f39b8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:54.146682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1aa69-3d27-4095-bc6a-3ff69f39b8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:54.147084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1aa69-3d27-4095-bc6a-3ff69f39b8b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:54.158299 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1aa69-3d27-4095-bc6a-3ff69f39b8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:54.159909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1aa69-3d27-4095-bc6a-3ff69f39b8b9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7fc1aa69-3d27-4095-bc6a-3ff69f39b8b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:25:54.178445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:54.179547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:54.180624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:25:54.180624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:25:54.180624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:54.180624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:25:54.180624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 613/1215] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:54.185679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d80bc-a5b3-4520-9983-6889fc7fba37 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:54.260284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:54.303597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d80bc-a5b3-4520-9983-6889fc7fba37 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:54.339330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b5d62-179a-4042-8d04-bfb8a4757d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:54.340001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b5d62-179a-4042-8d04-bfb8a4757d0d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:54.340280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b5d62-179a-4042-8d04-bfb8a4757d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:54.340553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b5d62-179a-4042-8d04-bfb8a4757d0d None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:54.341310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b5d62-179a-4042-8d04-bfb8a4757d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:54.341614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b5d62-179a-4042-8d04-bfb8a4757d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:54.343009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b5d62-179a-4042-8d04-bfb8a4757d0d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:54.355888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 603/1216] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:54.361305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1aa69-3d27-4095-bc6a-3ff69f39b8b9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:54.436966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:54.472318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1aa69-3d27-4095-bc6a-3ff69f39b8b9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:54.506841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce22fe-9e07-4e19-8f0a-37f0b5eb4158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:54.507639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce22fe-9e07-4e19-8f0a-37f0b5eb4158 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:54.507917 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce22fe-9e07-4e19-8f0a-37f0b5eb4158 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:54.508193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce22fe-9e07-4e19-8f0a-37f0b5eb4158 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:54.515111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce22fe-9e07-4e19-8f0a-37f0b5eb4158 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:54.516657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce22fe-9e07-4e19-8f0a-37f0b5eb4158 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b9ce22fe-9e07-4e19-8f0a-37f0b5eb4158 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:25:54.530996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:54.532181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:54.533234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:25:54.533234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:25:54.533234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:25:54.533234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:25:54.533234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:25:54.533234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:25:54.533234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:25:54.533234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:25:54.533234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 614/1217] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:54.538231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b5d62-179a-4042-8d04-bfb8a4757d0d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:54.646544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b5d62-179a-4042-8d04-bfb8a4757d0d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:54.674830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:54.689489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed25bc-6a6a-40e4-aa22-641283ebfe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:54.690191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed25bc-6a6a-40e4-aa22-641283ebfe80 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:54.690473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed25bc-6a6a-40e4-aa22-641283ebfe80 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:54.690749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed25bc-6a6a-40e4-aa22-641283ebfe80 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:54.691545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed25bc-6a6a-40e4-aa22-641283ebfe80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:54.691847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed25bc-6a6a-40e4-aa22-641283ebfe80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:54.693314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed25bc-6a6a-40e4-aa22-641283ebfe80 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:54.706439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 604/1218] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:25:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:54.712891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce22fe-9e07-4e19-8f0a-37f0b5eb4158 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:54.830625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce22fe-9e07-4e19-8f0a-37f0b5eb4158 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:54.864118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:54.870923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1c13f-7a6a-49a6-8457-273d99d899d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:54.871853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1c13f-7a6a-49a6-8457-273d99d899d1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:54.872366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1c13f-7a6a-49a6-8457-273d99d899d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:54.872786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1c13f-7a6a-49a6-8457-273d99d899d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:54.873772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1c13f-7a6a-49a6-8457-273d99d899d1 None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=9373333a1fd64852a138b3f5913168c1) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:54.874090 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1c13f-7a6a-49a6-8457-273d99d899d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:54.875664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1c13f-7a6a-49a6-8457-273d99d899d1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:54.881704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 615/1219] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:25:54 2022] GET /identity/v3/role_assignments?user.id=9373333a1fd64852a138b3f5913168c1&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:56.967831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12b11b-33da-4973-93b4-f35154665212 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:56.968438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12b11b-33da-4973-93b4-f35154665212 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:56.968560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12b11b-33da-4973-93b4-f35154665212 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:56.970011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 605/1220] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:25:56.975837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d56aa-fa27-44ba-ae3b-a78b930467a6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:56.976259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d56aa-fa27-44ba-ae3b-a78b930467a6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:56.976366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d56aa-fa27-44ba-ae3b-a78b930467a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:56.995204 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-868d56aa-fa27-44ba-ae3b-a78b930467a6 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:25:56.996440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-868d56aa-fa27-44ba-ae3b-a78b930467a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:25:57.071265 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 616/1221] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:25:57.097445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12b11b-33da-4973-93b4-f35154665212 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:57.216088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12b11b-33da-4973-93b4-f35154665212 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:57.256268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1f16a-0596-4c0c-9a80-5049d2bbefbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:57.257027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1f16a-0596-4c0c-9a80-5049d2bbefbf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:57.257310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1f16a-0596-4c0c-9a80-5049d2bbefbf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:57.257588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1f16a-0596-4c0c-9a80-5049d2bbefbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:57.270775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1f16a-0596-4c0c-9a80-5049d2bbefbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8ad1f16a-0596-4c0c-9a80-5049d2bbefbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:25:57.278550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:57.279678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:25:57.280712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 606/1222] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:25:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:57.289226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d56aa-fa27-44ba-ae3b-a78b930467a6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:57.397025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d56aa-fa27-44ba-ae3b-a78b930467a6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:57.424552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:57.441621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876b8db-c5d9-42b5-99ba-0dcbeb6b95d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:57.442318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876b8db-c5d9-42b5-99ba-0dcbeb6b95d8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:57.442600 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876b8db-c5d9-42b5-99ba-0dcbeb6b95d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:57.442872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876b8db-c5d9-42b5-99ba-0dcbeb6b95d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:57.443835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876b8db-c5d9-42b5-99ba-0dcbeb6b95d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:57.444217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876b8db-c5d9-42b5-99ba-0dcbeb6b95d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:57.446014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876b8db-c5d9-42b5-99ba-0dcbeb6b95d8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:57.463338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 617/1223] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:25:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:57.464520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1f16a-0596-4c0c-9a80-5049d2bbefbf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:57.517997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:57.580587 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1f16a-0596-4c0c-9a80-5049d2bbefbf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:57.617210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e8871-600d-41cb-b7e4-ffc59f05f279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:57.618152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e8871-600d-41cb-b7e4-ffc59f05f279 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:57.618529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e8871-600d-41cb-b7e4-ffc59f05f279 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:57.618912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e8871-600d-41cb-b7e4-ffc59f05f279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:57.630160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e8871-600d-41cb-b7e4-ffc59f05f279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:57.631624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e8871-600d-41cb-b7e4-ffc59f05f279 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-541e8871-600d-41cb-b7e4-ffc59f05f279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:25:57.649134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:57.650576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:57.651681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:25:57.651681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:25:57.651681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:57.651681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:25:57.651681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 607/1224] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:57.656669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876b8db-c5d9-42b5-99ba-0dcbeb6b95d8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:57.766488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876b8db-c5d9-42b5-99ba-0dcbeb6b95d8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:57.802685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5401d6e-862d-412a-94f1-f246441c3262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:57.803303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5401d6e-862d-412a-94f1-f246441c3262 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:57.803585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5401d6e-862d-412a-94f1-f246441c3262 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:57.803859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5401d6e-862d-412a-94f1-f246441c3262 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:57.804579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5401d6e-862d-412a-94f1-f246441c3262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:57.804918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5401d6e-862d-412a-94f1-f246441c3262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:57.806290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5401d6e-862d-412a-94f1-f246441c3262 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:57.818070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 618/1225] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:25:57.824074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e8871-600d-41cb-b7e4-ffc59f05f279 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:57.925967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:57.935197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e8871-600d-41cb-b7e4-ffc59f05f279 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:57.969840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0363f52-43c1-44ed-9ceb-2ebec7d375c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:57.970654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0363f52-43c1-44ed-9ceb-2ebec7d375c3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:57.970933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0363f52-43c1-44ed-9ceb-2ebec7d375c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:57.971200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0363f52-43c1-44ed-9ceb-2ebec7d375c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:57.979306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0363f52-43c1-44ed-9ceb-2ebec7d375c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:57.980779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0363f52-43c1-44ed-9ceb-2ebec7d375c3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b0363f52-43c1-44ed-9ceb-2ebec7d375c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:25:57.995683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:57.996845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:25:57.998056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:25:58.003569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 608/1226] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:25:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:58.003686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5401d6e-862d-412a-94f1-f246441c3262 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:58.052102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:58.108287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5401d6e-862d-412a-94f1-f246441c3262 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:58.144152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a08692-9f83-43a6-aa23-055c3ef2a356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:58.144897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a08692-9f83-43a6-aa23-055c3ef2a356 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:58.145165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a08692-9f83-43a6-aa23-055c3ef2a356 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:58.145439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a08692-9f83-43a6-aa23-055c3ef2a356 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:58.146274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a08692-9f83-43a6-aa23-055c3ef2a356 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:58.146581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a08692-9f83-43a6-aa23-055c3ef2a356 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:58.148027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a08692-9f83-43a6-aa23-055c3ef2a356 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:58.166725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 619/1227] 10.222.0.69 () {56 vars in 1143 bytes} [Fri Jun 24 10:25:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:58.172472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0363f52-43c1-44ed-9ceb-2ebec7d375c3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:58.281389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0363f52-43c1-44ed-9ceb-2ebec7d375c3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:58.317217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d40f5-c26c-4ec5-b37b-14d1fe2a50b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:58.318082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d40f5-c26c-4ec5-b37b-14d1fe2a50b6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:58.318366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d40f5-c26c-4ec5-b37b-14d1fe2a50b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:58.318719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d40f5-c26c-4ec5-b37b-14d1fe2a50b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:58.332019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d40f5-c26c-4ec5-b37b-14d1fe2a50b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:58.334211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d40f5-c26c-4ec5-b37b-14d1fe2a50b6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5e6d40f5-c26c-4ec5-b37b-14d1fe2a50b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:25:58.352442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:58.353602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:58.354680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:25:58.354680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:25:58.354680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:25:58.354680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:25:58.354680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 609/1228] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:25:58.359632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a08692-9f83-43a6-aa23-055c3ef2a356 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:58.465959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a08692-9f83-43a6-aa23-055c3ef2a356 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:58.468652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:58.504117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70648174-9c63-4f5b-91a7-1f223e85a651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:58.504812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70648174-9c63-4f5b-91a7-1f223e85a651 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:58.505095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70648174-9c63-4f5b-91a7-1f223e85a651 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:58.505410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70648174-9c63-4f5b-91a7-1f223e85a651 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:58.506199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70648174-9c63-4f5b-91a7-1f223e85a651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:58.506501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70648174-9c63-4f5b-91a7-1f223e85a651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:58.507937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70648174-9c63-4f5b-91a7-1f223e85a651 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:58.519951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 620/1229] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:58.525940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d40f5-c26c-4ec5-b37b-14d1fe2a50b6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:58.536394 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:58.634027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d40f5-c26c-4ec5-b37b-14d1fe2a50b6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:58.669138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e30f8-8b64-4dfc-8d9b-275bc4f5b546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:58.670087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e30f8-8b64-4dfc-8d9b-275bc4f5b546 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:58.670370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e30f8-8b64-4dfc-8d9b-275bc4f5b546 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:58.670651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e30f8-8b64-4dfc-8d9b-275bc4f5b546 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:58.678252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e30f8-8b64-4dfc-8d9b-275bc4f5b546 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:58.679865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e30f8-8b64-4dfc-8d9b-275bc4f5b546 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-fd2e30f8-8b64-4dfc-8d9b-275bc4f5b546 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:25:58.696420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:25:58.697553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:25:58.698738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:25:58.698738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:25:58.698738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:25:58.698738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:25:58.698738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:25:58.698738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:25:58.698738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:25:58.698738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:25:58.698738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 610/1230] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:25:58.703903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70648174-9c63-4f5b-91a7-1f223e85a651 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:58.824610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70648174-9c63-4f5b-91a7-1f223e85a651 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:58.862904 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306c14a-64d0-4161-8ce8-7fb72902557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:58.863638 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306c14a-64d0-4161-8ce8-7fb72902557f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:58.863952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306c14a-64d0-4161-8ce8-7fb72902557f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:58.864231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306c14a-64d0-4161-8ce8-7fb72902557f None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:58.865063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306c14a-64d0-4161-8ce8-7fb72902557f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:25:58.865378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306c14a-64d0-4161-8ce8-7fb72902557f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:58.866834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306c14a-64d0-4161-8ce8-7fb72902557f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:58.880193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 621/1231] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:25:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:25:58.886607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e30f8-8b64-4dfc-8d9b-275bc4f5b546 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:25:58.959241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:58.996967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e30f8-8b64-4dfc-8d9b-275bc4f5b546 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:25:59.034041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd2b9e-4a66-4ca5-b71a-744e295905a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:25:59.034968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd2b9e-4a66-4ca5-b71a-744e295905a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:59.035250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd2b9e-4a66-4ca5-b71a-744e295905a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:59.035530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd2b9e-4a66-4ca5-b71a-744e295905a2 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/9373333a1fd64852a138b3f5913168c1/roles/6b7e9f2fcda040b18213464589952d67` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:59.052159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd2b9e-4a66-4ca5-b71a-744e295905a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=9373333a1fd64852a138b3f5913168c1, role_id=6b7e9f2fcda040b18213464589952d67)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:25:59.053637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd2b9e-4a66-4ca5-b71a-744e295905a2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:25:59.068803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 611/1232] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:25:58 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/9373333a1fd64852a138b3f5913168c1/roles/6b7e9f2fcda040b18213464589952d67 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:01.165218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba277f8-f4e1-493b-bd23-a1d174d93638 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:01.165772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba277f8-f4e1-493b-bd23-a1d174d93638 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:01.165772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba277f8-f4e1-493b-bd23-a1d174d93638 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:01.167805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 622/1233] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:26:01.173177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7ec9e-59e8-4718-9759-f26ccc169829 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:01.173663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7ec9e-59e8-4718-9759-f26ccc169829 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:01.173777 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7ec9e-59e8-4718-9759-f26ccc169829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:01.191669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6fa7ec9e-59e8-4718-9759-f26ccc169829 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:01.193213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6fa7ec9e-59e8-4718-9759-f26ccc169829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:01.266463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 612/1234] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:26:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:26:01.291114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba277f8-f4e1-493b-bd23-a1d174d93638 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:01.328072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:01.420004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba277f8-f4e1-493b-bd23-a1d174d93638 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:01.461248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3dccb-e846-490b-b0a8-63b63c38644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:01.462140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3dccb-e846-490b-b0a8-63b63c38644c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:01.462419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3dccb-e846-490b-b0a8-63b63c38644c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:01.462696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3dccb-e846-490b-b0a8-63b63c38644c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:01.475074 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3dccb-e846-490b-b0a8-63b63c38644c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-57d3dccb-e846-490b-b0a8-63b63c38644c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:26:01.482795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:01.483758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:26:01.484604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:01.488912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 623/1235] 10.222.0.69 () {56 vars in 1102 bytes} [Fri Jun 24 10:26:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:01.490642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7ec9e-59e8-4718-9759-f26ccc169829 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:01.543165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:01.603533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7ec9e-59e8-4718-9759-f26ccc169829 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:01.643160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c594c-ccf9-40b7-9314-6eb175a55817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:01.643875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c594c-ccf9-40b7-9314-6eb175a55817 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:01.644131 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c594c-ccf9-40b7-9314-6eb175a55817 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:01.644406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c594c-ccf9-40b7-9314-6eb175a55817 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:01.645187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c594c-ccf9-40b7-9314-6eb175a55817 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:01.645489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c594c-ccf9-40b7-9314-6eb175a55817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:01.646865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c594c-ccf9-40b7-9314-6eb175a55817 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:01.655758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 613/1236] 10.222.0.69 () {56 vars in 1103 bytes} [Fri Jun 24 10:26:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:26:01.661722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3dccb-e846-490b-b0a8-63b63c38644c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:01.753543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:01.769712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3dccb-e846-490b-b0a8-63b63c38644c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:01.806294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10898836-b266-4d15-9d76-a37445ccc99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:01.807133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10898836-b266-4d15-9d76-a37445ccc99d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:01.807417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10898836-b266-4d15-9d76-a37445ccc99d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:01.807700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10898836-b266-4d15-9d76-a37445ccc99d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:01.819215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10898836-b266-4d15-9d76-a37445ccc99d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:01.821069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10898836-b266-4d15-9d76-a37445ccc99d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-10898836-b266-4d15-9d76-a37445ccc99d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:26:01.840250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:01.841241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:01.842148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:26:01.842148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:26:01.842148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:01.842148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:01.842148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 624/1237] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:01.849064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c594c-ccf9-40b7-9314-6eb175a55817 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:01.956400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c594c-ccf9-40b7-9314-6eb175a55817 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:01.968038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:01.996243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad6406-956c-4626-b0a2-d6e01c0105a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:01.996956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad6406-956c-4626-b0a2-d6e01c0105a6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:01.997241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad6406-956c-4626-b0a2-d6e01c0105a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:01.997518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad6406-956c-4626-b0a2-d6e01c0105a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:01.998271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad6406-956c-4626-b0a2-d6e01c0105a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:01.998571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad6406-956c-4626-b0a2-d6e01c0105a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:02.000135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad6406-956c-4626-b0a2-d6e01c0105a6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:02.013132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 614/1238] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:02.018724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10898836-b266-4d15-9d76-a37445ccc99d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:02.125955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10898836-b266-4d15-9d76-a37445ccc99d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:02.161370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85bc11-cd2b-41b9-8b11-f29426a68189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:02.162192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85bc11-cd2b-41b9-8b11-f29426a68189 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:02.162480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85bc11-cd2b-41b9-8b11-f29426a68189 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:02.162751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85bc11-cd2b-41b9-8b11-f29426a68189 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:02.165723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:02.172496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85bc11-cd2b-41b9-8b11-f29426a68189 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:02.174235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85bc11-cd2b-41b9-8b11-f29426a68189 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-6b85bc11-cd2b-41b9-8b11-f29426a68189 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:26:02.188936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:02.189974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:02.191119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 625/1239] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:02.196604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad6406-956c-4626-b0a2-d6e01c0105a6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:02.317248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad6406-956c-4626-b0a2-d6e01c0105a6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:02.354802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e8c0e-da14-46a2-b3f4-1cbf58a5d829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:02.355548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e8c0e-da14-46a2-b3f4-1cbf58a5d829 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:02.355846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e8c0e-da14-46a2-b3f4-1cbf58a5d829 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:02.356126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e8c0e-da14-46a2-b3f4-1cbf58a5d829 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:02.357091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e8c0e-da14-46a2-b3f4-1cbf58a5d829 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:02.357404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e8c0e-da14-46a2-b3f4-1cbf58a5d829 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:02.358957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e8c0e-da14-46a2-b3f4-1cbf58a5d829 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:02.378600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 615/1240] 10.222.0.69 () {56 vars in 1143 bytes} [Fri Jun 24 10:26:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:02.384206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85bc11-cd2b-41b9-8b11-f29426a68189 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:02.493976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85bc11-cd2b-41b9-8b11-f29426a68189 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:02.535363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a38c19-e965-4404-b734-551749a44781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:02.536019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38c19-e965-4404-b734-551749a44781 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:02.536303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38c19-e965-4404-b734-551749a44781 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:02.536581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38c19-e965-4404-b734-551749a44781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:02.604991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:02.612337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a38c19-e965-4404-b734-551749a44781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:02.614010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a38c19-e965-4404-b734-551749a44781 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:02.718826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 626/1241] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:26:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:26:02.725071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e8c0e-da14-46a2-b3f4-1cbf58a5d829 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:02.754920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:02.831806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e8c0e-da14-46a2-b3f4-1cbf58a5d829 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:02.872473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f55af-3df2-40ea-93c6-6a938cf97bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:02.873235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f55af-3df2-40ea-93c6-6a938cf97bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:02.873556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f55af-3df2-40ea-93c6-6a938cf97bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:02.873876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f55af-3df2-40ea-93c6-6a938cf97bb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:02.884712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f55af-3df2-40ea-93c6-6a938cf97bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:02.886352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f55af-3df2-40ea-93c6-6a938cf97bb2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-510f55af-3df2-40ea-93c6-6a938cf97bb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:26:02.911463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:02.912575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:02.918176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:26:02.918176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:26:02.918176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:02.918176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:02.918176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 616/1242] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:26:02.919188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a38c19-e965-4404-b734-551749a44781 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:03.025152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:03.032648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a38c19-e965-4404-b734-551749a44781 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:03.072541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc2a6d-12b8-456e-8636-748a6f572bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:03.073349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc2a6d-12b8-456e-8636-748a6f572bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:03.073668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc2a6d-12b8-456e-8636-748a6f572bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:03.074006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc2a6d-12b8-456e-8636-748a6f572bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:03.074740 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc2a6d-12b8-456e-8636-748a6f572bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:03.075050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc2a6d-12b8-456e-8636-748a6f572bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:03.076433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc2a6d-12b8-456e-8636-748a6f572bd5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:03.089693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 627/1243] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:03.096363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f55af-3df2-40ea-93c6-6a938cf97bb2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:03.190611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:03.219758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f55af-3df2-40ea-93c6-6a938cf97bb2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:03.264479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470431e3-bd7b-47c0-8d58-0094f33dba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:03.265403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470431e3-bd7b-47c0-8d58-0094f33dba80 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:03.265793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470431e3-bd7b-47c0-8d58-0094f33dba80 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:03.266226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470431e3-bd7b-47c0-8d58-0094f33dba80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:03.274032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470431e3-bd7b-47c0-8d58-0094f33dba80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:03.275486 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470431e3-bd7b-47c0-8d58-0094f33dba80 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-470431e3-bd7b-47c0-8d58-0094f33dba80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:26:03.291476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:03.292506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:03.293427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:26:03.293427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:26:03.293427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:26:03.293427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:26:03.293427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:26:03.293427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:26:03.293427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:26:03.293427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:03.293427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 617/1244] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:26:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:03.302142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc2a6d-12b8-456e-8636-748a6f572bd5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:03.404937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc2a6d-12b8-456e-8636-748a6f572bd5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:03.438641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106692a5-14e2-4ef4-92ff-71c98d2ffe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:03.439311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106692a5-14e2-4ef4-92ff-71c98d2ffe06 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:03.439590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106692a5-14e2-4ef4-92ff-71c98d2ffe06 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:03.439864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106692a5-14e2-4ef4-92ff-71c98d2ffe06 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:03.440792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106692a5-14e2-4ef4-92ff-71c98d2ffe06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:03.441098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106692a5-14e2-4ef4-92ff-71c98d2ffe06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:03.442508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106692a5-14e2-4ef4-92ff-71c98d2ffe06 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:03.458639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 628/1245] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:26:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:03.465448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470431e3-bd7b-47c0-8d58-0094f33dba80 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:03.582413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470431e3-bd7b-47c0-8d58-0094f33dba80 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:03.619161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee92c1-d545-4dbd-84bc-d0bdecadf644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:03.619809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee92c1-d545-4dbd-84bc-d0bdecadf644 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:03.620102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee92c1-d545-4dbd-84bc-d0bdecadf644 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:03.620356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee92c1-d545-4dbd-84bc-d0bdecadf644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:03.621301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee92c1-d545-4dbd-84bc-d0bdecadf644 None admin] RBAC: Adding query filter params (role.id=6b7e9f2fcda040b18213464589952d67, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=9373333a1fd64852a138b3f5913168c1) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:03.621641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee92c1-d545-4dbd-84bc-d0bdecadf644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:03.623198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee92c1-d545-4dbd-84bc-d0bdecadf644 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:03.629775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 618/1246] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:26:03 2022] GET /identity/v3/role_assignments?user.id=9373333a1fd64852a138b3f5913168c1&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=6b7e9f2fcda040b18213464589952d67 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:05.817089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922da13-74b3-482e-8e0d-6b344d9406de None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:05.817603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922da13-74b3-482e-8e0d-6b344d9406de None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:05.817603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922da13-74b3-482e-8e0d-6b344d9406de None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:05.818979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 629/1247] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:26:05.824689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5add88-b7cd-4a6c-9b2e-7778cb69da7a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:05.825185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5add88-b7cd-4a6c-9b2e-7778cb69da7a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:05.825306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5add88-b7cd-4a6c-9b2e-7778cb69da7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:05.831122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:05.844485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3f5add88-b7cd-4a6c-9b2e-7778cb69da7a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:05.845719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3f5add88-b7cd-4a6c-9b2e-7778cb69da7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:05.924991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 619/1248] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:26:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:26:05.949810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922da13-74b3-482e-8e0d-6b344d9406de None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:05.970945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:06.069972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922da13-74b3-482e-8e0d-6b344d9406de None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:06.108296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64e9a4-852b-40b5-9206-e4e1ad808988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:06.109576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64e9a4-852b-40b5-9206-e4e1ad808988 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:06.109969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64e9a4-852b-40b5-9206-e4e1ad808988 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:06.110286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64e9a4-852b-40b5-9206-e4e1ad808988 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:06.121814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64e9a4-852b-40b5-9206-e4e1ad808988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-8c64e9a4-852b-40b5-9206-e4e1ad808988 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:26:06.130458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:06.131594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:06.132617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 630/1249] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:06.138175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5add88-b7cd-4a6c-9b2e-7778cb69da7a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:06.255674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5add88-b7cd-4a6c-9b2e-7778cb69da7a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:06.289827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:06.296089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3d3db-6946-4d90-81d3-2d4bbe6eb6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:06.296844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3d3db-6946-4d90-81d3-2d4bbe6eb6ae None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:06.297119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3d3db-6946-4d90-81d3-2d4bbe6eb6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:06.297393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3d3db-6946-4d90-81d3-2d4bbe6eb6ae None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:06.298175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3d3db-6946-4d90-81d3-2d4bbe6eb6ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:06.298610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3d3db-6946-4d90-81d3-2d4bbe6eb6ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:06.300123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3d3db-6946-4d90-81d3-2d4bbe6eb6ae None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:06.310314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 620/1250] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:26:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:06.316324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64e9a4-852b-40b5-9206-e4e1ad808988 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:06.397579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:06.422221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64e9a4-852b-40b5-9206-e4e1ad808988 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:06.461535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f21855-2365-4e97-a006-d8ceb51d5769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:06.462401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21855-2365-4e97-a006-d8ceb51d5769 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:06.462709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21855-2365-4e97-a006-d8ceb51d5769 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:06.463052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21855-2365-4e97-a006-d8ceb51d5769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:06.476048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f21855-2365-4e97-a006-d8ceb51d5769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:06.477531 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f21855-2365-4e97-a006-d8ceb51d5769 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a5f21855-2365-4e97-a006-d8ceb51d5769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:26:06.497678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:06.498901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:06.499870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:26:06.499870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:26:06.499870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:06.499870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:06.499870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 631/1251] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:26:06.505881 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3d3db-6946-4d90-81d3-2d4bbe6eb6ae None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:06.623890 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3d3db-6946-4d90-81d3-2d4bbe6eb6ae None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:06.661361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab4809-74d4-4954-9c06-26e18f179048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:06.662033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab4809-74d4-4954-9c06-26e18f179048 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:06.662311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab4809-74d4-4954-9c06-26e18f179048 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:06.662581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab4809-74d4-4954-9c06-26e18f179048 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:06.663307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab4809-74d4-4954-9c06-26e18f179048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:06.663611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab4809-74d4-4954-9c06-26e18f179048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:06.665032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab4809-74d4-4954-9c06-26e18f179048 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:06.677844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 621/1252] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:26:06.684676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f21855-2365-4e97-a006-d8ceb51d5769 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:06.810003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f21855-2365-4e97-a006-d8ceb51d5769 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:06.839799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:06.849383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e36be-1350-4e0e-84eb-e56940ae42a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:06.850755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e36be-1350-4e0e-84eb-e56940ae42a9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:06.851123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e36be-1350-4e0e-84eb-e56940ae42a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:06.851408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e36be-1350-4e0e-84eb-e56940ae42a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:06.864090 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e36be-1350-4e0e-84eb-e56940ae42a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:06.865692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e36be-1350-4e0e-84eb-e56940ae42a9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d17e36be-1350-4e0e-84eb-e56940ae42a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:26:06.880451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:06.881725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:06.882848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 632/1253] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:06.888389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab4809-74d4-4954-9c06-26e18f179048 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:06.957657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:07.011073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab4809-74d4-4954-9c06-26e18f179048 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:07.050273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615e07f-0d9e-4062-b7c7-f232970c2240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:07.050962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615e07f-0d9e-4062-b7c7-f232970c2240 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:07.051262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615e07f-0d9e-4062-b7c7-f232970c2240 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:07.051527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615e07f-0d9e-4062-b7c7-f232970c2240 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:07.052432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615e07f-0d9e-4062-b7c7-f232970c2240 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:07.052699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615e07f-0d9e-4062-b7c7-f232970c2240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:07.054149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615e07f-0d9e-4062-b7c7-f232970c2240 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:07.076410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 622/1254] 10.222.0.69 () {56 vars in 1143 bytes} [Fri Jun 24 10:26:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:07.082814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e36be-1350-4e0e-84eb-e56940ae42a9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:07.195520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e36be-1350-4e0e-84eb-e56940ae42a9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:07.236180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac2ae3-fa24-44bd-a59d-cd4e701517bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:07.236875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac2ae3-fa24-44bd-a59d-cd4e701517bd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:07.237215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac2ae3-fa24-44bd-a59d-cd4e701517bd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:07.237547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac2ae3-fa24-44bd-a59d-cd4e701517bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:07.304193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:07.317692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac2ae3-fa24-44bd-a59d-cd4e701517bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:07.319243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac2ae3-fa24-44bd-a59d-cd4e701517bd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:07.431606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 633/1255] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:26:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:26:07.438085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615e07f-0d9e-4062-b7c7-f232970c2240 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:07.556148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615e07f-0d9e-4062-b7c7-f232970c2240 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:07.572185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:07.598697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad568b8-4792-46e8-9568-62f52922563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:07.599686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad568b8-4792-46e8-9568-62f52922563a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:07.599970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad568b8-4792-46e8-9568-62f52922563a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:07.600276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad568b8-4792-46e8-9568-62f52922563a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:07.613476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad568b8-4792-46e8-9568-62f52922563a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:07.614943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad568b8-4792-46e8-9568-62f52922563a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-cad568b8-4792-46e8-9568-62f52922563a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:26:07.636565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:07.637602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:07.638499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:26:07.638499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:26:07.638499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:07.638499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:07.638499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 623/1256] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:07.644157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac2ae3-fa24-44bd-a59d-cd4e701517bd None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:07.751513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:07.768089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac2ae3-fa24-44bd-a59d-cd4e701517bd None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:07.814263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48b558-4a51-4cce-b29a-08bddd4d8b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:07.815020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48b558-4a51-4cce-b29a-08bddd4d8b08 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:07.815310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48b558-4a51-4cce-b29a-08bddd4d8b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:07.815628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48b558-4a51-4cce-b29a-08bddd4d8b08 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:07.816539 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48b558-4a51-4cce-b29a-08bddd4d8b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:07.816894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48b558-4a51-4cce-b29a-08bddd4d8b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:07.818521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48b558-4a51-4cce-b29a-08bddd4d8b08 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:07.832159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 634/1257] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:26:07.838658 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad568b8-4792-46e8-9568-62f52922563a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:07.950379 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad568b8-4792-46e8-9568-62f52922563a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:07.986760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484ca5b-ae95-4ec5-b666-1f374a2183f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:07.987566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484ca5b-ae95-4ec5-b666-1f374a2183f4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:07.987847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484ca5b-ae95-4ec5-b666-1f374a2183f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:07.988166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484ca5b-ae95-4ec5-b666-1f374a2183f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:07.995812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484ca5b-ae95-4ec5-b666-1f374a2183f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:07.997419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484ca5b-ae95-4ec5-b666-1f374a2183f4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6484ca5b-ae95-4ec5-b666-1f374a2183f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:26:08.012389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:08.013533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:08.014542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:26:08.014542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:26:08.014542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:26:08.014542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:26:08.014542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:26:08.014542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:26:08.014542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:26:08.014542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:08.014542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 624/1258] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:08.019650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48b558-4a51-4cce-b29a-08bddd4d8b08 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:08.134433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48b558-4a51-4cce-b29a-08bddd4d8b08 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:08.170489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6a89f-ffe3-4841-bfbe-759c98362b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:08.171193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6a89f-ffe3-4841-bfbe-759c98362b5e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:08.171489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6a89f-ffe3-4841-bfbe-759c98362b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:08.171811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6a89f-ffe3-4841-bfbe-759c98362b5e None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:08.172677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6a89f-ffe3-4841-bfbe-759c98362b5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:08.173028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6a89f-ffe3-4841-bfbe-759c98362b5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:08.174449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6a89f-ffe3-4841-bfbe-759c98362b5e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:08.186776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 635/1259] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:08.193296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484ca5b-ae95-4ec5-b666-1f374a2183f4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:08.221458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:08.321415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484ca5b-ae95-4ec5-b666-1f374a2183f4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:08.372772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bf66d-593f-4822-ac18-505ed0cd0062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:08.373461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bf66d-593f-4822-ac18-505ed0cd0062 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:08.373774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bf66d-593f-4822-ac18-505ed0cd0062 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:08.374015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bf66d-593f-4822-ac18-505ed0cd0062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:08.375034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bf66d-593f-4822-ac18-505ed0cd0062 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=9373333a1fd64852a138b3f5913168c1) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:08.375573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bf66d-593f-4822-ac18-505ed0cd0062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:08.377094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bf66d-593f-4822-ac18-505ed0cd0062 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:08.382939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 625/1260] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:26:08 2022] GET /identity/v3/role_assignments?user.id=9373333a1fd64852a138b3f5913168c1&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:10.570852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de36fb8-005f-4f52-a09f-13f2684d0582 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:10.571387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de36fb8-005f-4f52-a09f-13f2684d0582 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:10.571387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de36fb8-005f-4f52-a09f-13f2684d0582 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:10.572789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 636/1261] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:26:10.581726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4be44-c3aa-4c6f-a98a-c7cc9d8b936c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:10.582410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4be44-c3aa-4c6f-a98a-c7cc9d8b936c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:10.582600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4be44-c3aa-4c6f-a98a-c7cc9d8b936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:10.604947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f6f4be44-c3aa-4c6f-a98a-c7cc9d8b936c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:10.606146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f6f4be44-c3aa-4c6f-a98a-c7cc9d8b936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:10.682263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 626/1262] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:26:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:26:10.713839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de36fb8-005f-4f52-a09f-13f2684d0582 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:10.725185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:10.819875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de36fb8-005f-4f52-a09f-13f2684d0582 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:10.858269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09055c8-01ac-4478-9b83-fa4b56995b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:10.859139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09055c8-01ac-4478-9b83-fa4b56995b03 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:10.859655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09055c8-01ac-4478-9b83-fa4b56995b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:10.859938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09055c8-01ac-4478-9b83-fa4b56995b03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:10.870644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09055c8-01ac-4478-9b83-fa4b56995b03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a09055c8-01ac-4478-9b83-fa4b56995b03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:26:10.878222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:10.879240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:10.880309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 637/1263] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:26:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:10.886287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4be44-c3aa-4c6f-a98a-c7cc9d8b936c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:10.909571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:11.001090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4be44-c3aa-4c6f-a98a-c7cc9d8b936c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.038565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28928830-85ec-4a1a-90f8-9fec0f720b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.039230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28928830-85ec-4a1a-90f8-9fec0f720b0c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.039510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28928830-85ec-4a1a-90f8-9fec0f720b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.039786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28928830-85ec-4a1a-90f8-9fec0f720b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.040534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28928830-85ec-4a1a-90f8-9fec0f720b0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:11.040884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28928830-85ec-4a1a-90f8-9fec0f720b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.042296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28928830-85ec-4a1a-90f8-9fec0f720b0c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.050806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 627/1264] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:26:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:11.057018 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09055c8-01ac-4478-9b83-fa4b56995b03 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.129523 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:11.164348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09055c8-01ac-4478-9b83-fa4b56995b03 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.198957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d14ffb-de1e-4f75-b85a-f738600ae499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.199908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14ffb-de1e-4f75-b85a-f738600ae499 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.200182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14ffb-de1e-4f75-b85a-f738600ae499 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.200470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14ffb-de1e-4f75-b85a-f738600ae499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.217958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d14ffb-de1e-4f75-b85a-f738600ae499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.219689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d14ffb-de1e-4f75-b85a-f738600ae499 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-71d14ffb-de1e-4f75-b85a-f738600ae499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:26:11.242132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:11.243209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:11.244697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:26:11.244697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:26:11.244697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:11.244697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:11.244697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 638/1265] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:11.250901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28928830-85ec-4a1a-90f8-9fec0f720b0c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.344623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:11.368084 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28928830-85ec-4a1a-90f8-9fec0f720b0c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.405328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1ec95-e9f6-49f9-afc3-a9831f363613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.406038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1ec95-e9f6-49f9-afc3-a9831f363613 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.406322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1ec95-e9f6-49f9-afc3-a9831f363613 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.406601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1ec95-e9f6-49f9-afc3-a9831f363613 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.407305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1ec95-e9f6-49f9-afc3-a9831f363613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:11.407610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1ec95-e9f6-49f9-afc3-a9831f363613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.408986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1ec95-e9f6-49f9-afc3-a9831f363613 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.420705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 628/1266] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:11.426891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d14ffb-de1e-4f75-b85a-f738600ae499 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.530525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d14ffb-de1e-4f75-b85a-f738600ae499 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.550360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:11.565804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957362c-2217-4a98-978d-f75648a7a9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.566595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957362c-2217-4a98-978d-f75648a7a9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.566885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957362c-2217-4a98-978d-f75648a7a9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.567159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957362c-2217-4a98-978d-f75648a7a9e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.574547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957362c-2217-4a98-978d-f75648a7a9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.575995 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957362c-2217-4a98-978d-f75648a7a9e3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0957362c-2217-4a98-978d-f75648a7a9e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:26:11.590541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:11.591656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:11.592586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 639/1267] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:11.597922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1ec95-e9f6-49f9-afc3-a9831f363613 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.703727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1ec95-e9f6-49f9-afc3-a9831f363613 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.740706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a4d6f-cc39-42ff-8459-fa987759f641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.741406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a4d6f-cc39-42ff-8459-fa987759f641 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.741689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a4d6f-cc39-42ff-8459-fa987759f641 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.741969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a4d6f-cc39-42ff-8459-fa987759f641 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.742816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a4d6f-cc39-42ff-8459-fa987759f641 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:11.743124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a4d6f-cc39-42ff-8459-fa987759f641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.744551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a4d6f-cc39-42ff-8459-fa987759f641 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.753409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:11.762318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 629/1268] 10.222.0.69 () {56 vars in 1143 bytes} [Fri Jun 24 10:26:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:11.768319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957362c-2217-4a98-978d-f75648a7a9e3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.877644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957362c-2217-4a98-978d-f75648a7a9e3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.912136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fafe6b-545f-4039-8020-5b760b684563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.912945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fafe6b-545f-4039-8020-5b760b684563 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.913225 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fafe6b-545f-4039-8020-5b760b684563 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.913506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fafe6b-545f-4039-8020-5b760b684563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.924877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fafe6b-545f-4039-8020-5b760b684563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.926331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fafe6b-545f-4039-8020-5b760b684563 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-13fafe6b-545f-4039-8020-5b760b684563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:26:11.943244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:11.944219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:11.945586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:26:11.945586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:26:11.945586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:11.945586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:11.945586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 640/1269] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:11.953542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a4d6f-cc39-42ff-8459-fa987759f641 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:12.065257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a4d6f-cc39-42ff-8459-fa987759f641 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:12.101951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10b82f-0a56-410b-b4a2-39bf50818212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:12.102604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10b82f-0a56-410b-b4a2-39bf50818212 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:12.102880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10b82f-0a56-410b-b4a2-39bf50818212 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:12.103151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10b82f-0a56-410b-b4a2-39bf50818212 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:12.103871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10b82f-0a56-410b-b4a2-39bf50818212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:12.104174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10b82f-0a56-410b-b4a2-39bf50818212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:12.105565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10b82f-0a56-410b-b4a2-39bf50818212 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:12.116928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 630/1270] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:12.125369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fafe6b-545f-4039-8020-5b760b684563 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:12.160300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:12.237208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fafe6b-545f-4039-8020-5b760b684563 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:12.274524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31272ed-2bb9-43ea-bcad-38b59996dc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:12.275323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31272ed-2bb9-43ea-bcad-38b59996dc27 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:12.275609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31272ed-2bb9-43ea-bcad-38b59996dc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:12.275888 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31272ed-2bb9-43ea-bcad-38b59996dc27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:12.282902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31272ed-2bb9-43ea-bcad-38b59996dc27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:12.284355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31272ed-2bb9-43ea-bcad-38b59996dc27 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b31272ed-2bb9-43ea-bcad-38b59996dc27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:26:12.298925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:12.299884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:12.300852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:26:12.300852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:26:12.300852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:26:12.300852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:26:12.300852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:26:12.300852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:26:12.300852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:26:12.300852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:12.300852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 641/1271] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:26:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:12.309156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10b82f-0a56-410b-b4a2-39bf50818212 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:12.381074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:12.427500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10b82f-0a56-410b-b4a2-39bf50818212 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:12.469534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa1703-5509-4e67-b477-0da9bbe24888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:12.470200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa1703-5509-4e67-b477-0da9bbe24888 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:12.470478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa1703-5509-4e67-b477-0da9bbe24888 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:12.470750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa1703-5509-4e67-b477-0da9bbe24888 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:12.471540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa1703-5509-4e67-b477-0da9bbe24888 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:12.471842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa1703-5509-4e67-b477-0da9bbe24888 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:12.473290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa1703-5509-4e67-b477-0da9bbe24888 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:12.485728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 631/1272] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:26:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:12.491854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31272ed-2bb9-43ea-bcad-38b59996dc27 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:12.603356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:12.613703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31272ed-2bb9-43ea-bcad-38b59996dc27 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:12.649495 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d12ec-7aa0-453b-9e33-49df3efa5182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:12.650376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d12ec-7aa0-453b-9e33-49df3efa5182 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:12.650659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d12ec-7aa0-453b-9e33-49df3efa5182 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:12.650930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d12ec-7aa0-453b-9e33-49df3efa5182 None admin] PATH_INFO: `/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/9373333a1fd64852a138b3f5913168c1/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:12.653147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d12ec-7aa0-453b-9e33-49df3efa5182 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbbd17a9ff41e98d76358258a07b80, user_id=9373333a1fd64852a138b3f5913168c1, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:12.654587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d12ec-7aa0-453b-9e33-49df3efa5182 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:12.671631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 642/1273] 10.222.0.69 () {58 vars in 1439 bytes} [Fri Jun 24 10:26:12 2022] PUT /identity/v3/projects/d6fbbd17a9ff41e98d76358258a07b80/users/9373333a1fd64852a138b3f5913168c1/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:14.705489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116724d-ab3f-485c-a899-e7f69c4db2c9 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:14.706029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116724d-ab3f-485c-a899-e7f69c4db2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:14.706391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116724d-ab3f-485c-a899-e7f69c4db2c9 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:14.709040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 632/1274] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:26:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:26:14.716845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245623ae-c8ca-48d4-83d5-7e76b2431e6d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:14.717318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245623ae-c8ca-48d4-83d5-7e76b2431e6d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:14.717410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245623ae-c8ca-48d4-83d5-7e76b2431e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:14.736363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-245623ae-c8ca-48d4-83d5-7e76b2431e6d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:14.737582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-245623ae-c8ca-48d4-83d5-7e76b2431e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:14.805599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 643/1275] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:26:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:26:14.831748 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116724d-ab3f-485c-a899-e7f69c4db2c9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:14.946346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116724d-ab3f-485c-a899-e7f69c4db2c9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:14.966595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:14.985778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf91889-e244-477e-9517-6326e358a2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:14.986568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf91889-e244-477e-9517-6326e358a2fa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:14.986854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf91889-e244-477e-9517-6326e358a2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:14.987133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf91889-e244-477e-9517-6326e358a2fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:14.998299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf91889-e244-477e-9517-6326e358a2fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5bf91889-e244-477e-9517-6326e358a2fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:26:15.006118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:15.007148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:15.008085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 633/1276] 10.222.0.69 () {56 vars in 1096 bytes} [Fri Jun 24 10:26:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:15.014415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245623ae-c8ca-48d4-83d5-7e76b2431e6d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:15.106693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:15.127017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245623ae-c8ca-48d4-83d5-7e76b2431e6d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:15.162528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb7b49-5c66-4a89-a21c-53716a83be12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:15.163228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb7b49-5c66-4a89-a21c-53716a83be12 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:15.163539 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb7b49-5c66-4a89-a21c-53716a83be12 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:15.163810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb7b49-5c66-4a89-a21c-53716a83be12 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:15.164563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb7b49-5c66-4a89-a21c-53716a83be12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:15.164902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb7b49-5c66-4a89-a21c-53716a83be12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:15.166286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb7b49-5c66-4a89-a21c-53716a83be12 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:15.174502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 644/1277] 10.222.0.69 () {56 vars in 1099 bytes} [Fri Jun 24 10:26:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:15.180671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf91889-e244-477e-9517-6326e358a2fa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:15.310686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf91889-e244-477e-9517-6326e358a2fa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:15.346610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670df67-82e5-485b-9712-4cb854a430c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:15.347398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670df67-82e5-485b-9712-4cb854a430c0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:15.347685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670df67-82e5-485b-9712-4cb854a430c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:15.347957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670df67-82e5-485b-9712-4cb854a430c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:15.359123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670df67-82e5-485b-9712-4cb854a430c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:15.360590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670df67-82e5-485b-9712-4cb854a430c0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d670df67-82e5-485b-9712-4cb854a430c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:26:15.378692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:15.379676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:15.380588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:26:15.380588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:26:15.380588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:15.380588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:15.380588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 634/1278] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:26:15.386210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb7b49-5c66-4a89-a21c-53716a83be12 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:15.501445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb7b49-5c66-4a89-a21c-53716a83be12 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:15.536852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb72c9-cd51-4357-bf28-ba07f9e00f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:15.537513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb72c9-cd51-4357-bf28-ba07f9e00f68 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:15.537806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb72c9-cd51-4357-bf28-ba07f9e00f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:15.538086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb72c9-cd51-4357-bf28-ba07f9e00f68 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:15.538839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb72c9-cd51-4357-bf28-ba07f9e00f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:15.539152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb72c9-cd51-4357-bf28-ba07f9e00f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:15.540535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb72c9-cd51-4357-bf28-ba07f9e00f68 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:15.553364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 645/1279] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:15.562582 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670df67-82e5-485b-9712-4cb854a430c0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:15.599644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:15.681171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670df67-82e5-485b-9712-4cb854a430c0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:15.718752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd82423-5c21-4d3a-be86-14811092af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:15.719744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd82423-5c21-4d3a-be86-14811092af07 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:15.720033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd82423-5c21-4d3a-be86-14811092af07 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:15.720307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd82423-5c21-4d3a-be86-14811092af07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:15.728057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd82423-5c21-4d3a-be86-14811092af07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:15.729563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd82423-5c21-4d3a-be86-14811092af07 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-dfd82423-5c21-4d3a-be86-14811092af07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:26:15.744068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:15.745162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:15.746220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 635/1280] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:15.751467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb72c9-cd51-4357-bf28-ba07f9e00f68 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:15.755320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:15.870836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb72c9-cd51-4357-bf28-ba07f9e00f68 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:15.913950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989a295-9063-4738-9d19-0bfcc107d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:15.914616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989a295-9063-4738-9d19-0bfcc107d771 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:15.914900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989a295-9063-4738-9d19-0bfcc107d771 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:15.915177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989a295-9063-4738-9d19-0bfcc107d771 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:15.916013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989a295-9063-4738-9d19-0bfcc107d771 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:15.916326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989a295-9063-4738-9d19-0bfcc107d771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:15.917794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989a295-9063-4738-9d19-0bfcc107d771 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:15.935607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 646/1281] 10.222.0.69 () {56 vars in 1143 bytes} [Fri Jun 24 10:26:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:15.941817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd82423-5c21-4d3a-be86-14811092af07 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:16.053368 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:16.064021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd82423-5c21-4d3a-be86-14811092af07 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:16.099924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a2616-b9da-4b85-9e6d-a40229005945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:16.100555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a2616-b9da-4b85-9e6d-a40229005945 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:16.100868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a2616-b9da-4b85-9e6d-a40229005945 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:16.101143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a2616-b9da-4b85-9e6d-a40229005945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:16.183107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a2616-b9da-4b85-9e6d-a40229005945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:16.184546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a2616-b9da-4b85-9e6d-a40229005945 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:16.287589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:16.304446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 636/1282] 10.222.0.69 () {58 vars in 1282 bytes} [Fri Jun 24 10:26:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:26:16.311551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989a295-9063-4738-9d19-0bfcc107d771 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:16.377264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:16.421506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989a295-9063-4738-9d19-0bfcc107d771 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:16.458079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ad9e0-ecdb-4103-b8b0-48e179578bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:16.459230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ad9e0-ecdb-4103-b8b0-48e179578bdb None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:16.459512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ad9e0-ecdb-4103-b8b0-48e179578bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:16.459783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ad9e0-ecdb-4103-b8b0-48e179578bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:16.472360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ad9e0-ecdb-4103-b8b0-48e179578bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:16.473883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ad9e0-ecdb-4103-b8b0-48e179578bdb None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-010ad9e0-ecdb-4103-b8b0-48e179578bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:26:16.492100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:16.493101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:16.494147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:26:16.494147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:26:16.494147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:26:16.494147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:16.494147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 647/1283] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:16.499522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a2616-b9da-4b85-9e6d-a40229005945 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:16.602784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a2616-b9da-4b85-9e6d-a40229005945 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:16.638644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca03c9a-1928-469c-a52e-33aaeb5425ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:16.639290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03c9a-1928-469c-a52e-33aaeb5425ad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:16.639572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03c9a-1928-469c-a52e-33aaeb5425ad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:16.639853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03c9a-1928-469c-a52e-33aaeb5425ad None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:16.640669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca03c9a-1928-469c-a52e-33aaeb5425ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:16.641103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca03c9a-1928-469c-a52e-33aaeb5425ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:16.642471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca03c9a-1928-469c-a52e-33aaeb5425ad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:16.653816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 637/1284] 10.222.0.69 () {56 vars in 1109 bytes} [Fri Jun 24 10:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:16.659955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ad9e0-ecdb-4103-b8b0-48e179578bdb None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:16.770791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ad9e0-ecdb-4103-b8b0-48e179578bdb None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:16.786220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:16.808170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68b553-3bd9-4b54-8742-fc7cd581b7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:16.808995 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68b553-3bd9-4b54-8742-fc7cd581b7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:16.809280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68b553-3bd9-4b54-8742-fc7cd581b7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:16.809554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68b553-3bd9-4b54-8742-fc7cd581b7e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:16.816581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68b553-3bd9-4b54-8742-fc7cd581b7e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:16.818068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68b553-3bd9-4b54-8742-fc7cd581b7e4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-be68b553-3bd9-4b54-8742-fc7cd581b7e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:26:16.832235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:16.833283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:16.834733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:26:16.834733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:26:16.834733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:26:16.834733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:26:16.834733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:26:16.834733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:26:16.834733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:26:16.834733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Jun 24 10:26:16.834733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 648/1285] 10.222.0.69 () {56 vars in 1111 bytes} [Fri Jun 24 10:26:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:16.840337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca03c9a-1928-469c-a52e-33aaeb5425ad None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:16.899338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:16.956434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca03c9a-1928-469c-a52e-33aaeb5425ad None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:16.992200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7e8bb-247a-4523-86eb-047f1c4ca645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:16.992873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7e8bb-247a-4523-86eb-047f1c4ca645 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:16.993162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7e8bb-247a-4523-86eb-047f1c4ca645 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:16.993429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7e8bb-247a-4523-86eb-047f1c4ca645 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:16.994224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7e8bb-247a-4523-86eb-047f1c4ca645 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:16.994531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7e8bb-247a-4523-86eb-047f1c4ca645 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:16.995966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7e8bb-247a-4523-86eb-047f1c4ca645 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:17.007967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 638/1286] 10.222.0.69 () {56 vars in 1148 bytes} [Fri Jun 24 10:26:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:17.014382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68b553-3bd9-4b54-8742-fc7cd581b7e4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:17.132673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68b553-3bd9-4b54-8742-fc7cd581b7e4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:17.168995 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac7ddb-56be-4639-bcea-34c75f10a4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:17.169643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac7ddb-56be-4639-bcea-34c75f10a4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:17.169925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac7ddb-56be-4639-bcea-34c75f10a4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:17.170200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac7ddb-56be-4639-bcea-34c75f10a4d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:17.171027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac7ddb-56be-4639-bcea-34c75f10a4d1 None admin] RBAC: Adding query filter params (role.id=fff8f769b9bf46018c80bd5d274e7888, scope.project.id=d6fbbd17a9ff41e98d76358258a07b80, user.id=9373333a1fd64852a138b3f5913168c1) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:17.171338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac7ddb-56be-4639-bcea-34c75f10a4d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:17.172806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac7ddb-56be-4639-bcea-34c75f10a4d1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:17.178855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 649/1287] 10.222.0.69 () {56 vars in 1374 bytes} [Fri Jun 24 10:26:17 2022] GET /identity/v3/role_assignments?user.id=9373333a1fd64852a138b3f5913168c1&scope.project.id=d6fbbd17a9ff41e98d76358258a07b80&role.id=fff8f769b9bf46018c80bd5d274e7888 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:19.285085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f7d2e-1475-41fc-b076-1135bb27ebca None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.285672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f7d2e-1475-41fc-b076-1135bb27ebca None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.285672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f7d2e-1475-41fc-b076-1135bb27ebca None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.287000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 639/1288] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:26:19.292623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272130a6-8f70-487b-8f9b-17df15f85026 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.293083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272130a6-8f70-487b-8f9b-17df15f85026 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.293172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272130a6-8f70-487b-8f9b-17df15f85026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.311472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-272130a6-8f70-487b-8f9b-17df15f85026 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:19.312702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-272130a6-8f70-487b-8f9b-17df15f85026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:19.342939 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:19.382904 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 650/1289] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:26:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:26:19.408142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f7d2e-1475-41fc-b076-1135bb27ebca None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.524766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f7d2e-1475-41fc-b076-1135bb27ebca None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.535584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:19.565166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c4d02-d750-4722-916d-d0f9e85b7f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.565964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c4d02-d750-4722-916d-d0f9e85b7f31 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.566243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c4d02-d750-4722-916d-d0f9e85b7f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.566515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c4d02-d750-4722-916d-d0f9e85b7f31 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.567204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c4d02-d750-4722-916d-d0f9e85b7f31 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.568565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c4d02-d750-4722-916d-d0f9e85b7f31 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-435c4d02-d750-4722-916d-d0f9e85b7f31 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:19.581071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:19.582324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:19.583357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 640/1290] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:26:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:19.588826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272130a6-8f70-487b-8f9b-17df15f85026 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.690424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272130a6-8f70-487b-8f9b-17df15f85026 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.727217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90723fa9-0693-428d-a7d0-7916b1af7ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.727882 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90723fa9-0693-428d-a7d0-7916b1af7ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.728160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90723fa9-0693-428d-a7d0-7916b1af7ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.728433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90723fa9-0693-428d-a7d0-7916b1af7ef3 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.729176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90723fa9-0693-428d-a7d0-7916b1af7ef3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:19.729486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90723fa9-0693-428d-a7d0-7916b1af7ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.730842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90723fa9-0693-428d-a7d0-7916b1af7ef3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.736391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 651/1291] 10.222.0.69 () {56 vars in 1116 bytes} [Fri Jun 24 10:26:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:26:19.745917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c4d02-d750-4722-916d-d0f9e85b7f31 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.873326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c4d02-d750-4722-916d-d0f9e85b7f31 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.908956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0752cc6-ea87-4f18-8866-17e6a64e8b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.909615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0752cc6-ea87-4f18-8866-17e6a64e8b52 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.909893 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0752cc6-ea87-4f18-8866-17e6a64e8b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.910167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0752cc6-ea87-4f18-8866-17e6a64e8b52 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.910876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0752cc6-ea87-4f18-8866-17e6a64e8b52 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:19.911185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0752cc6-ea87-4f18-8866-17e6a64e8b52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.912557 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0752cc6-ea87-4f18-8866-17e6a64e8b52 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.917394 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 641/1292] 10.222.0.69 () {56 vars in 1116 bytes} [Fri Jun 24 10:26:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:21.902440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5e47b-d339-4404-a7e5-cc7105d028de None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:21.903283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5e47b-d339-4404-a7e5-cc7105d028de None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:21.903283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5e47b-d339-4404-a7e5-cc7105d028de None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:21.905402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 652/1293] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:26:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:26:21.914207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99850566-2443-4cc0-9406-515a002dbc80 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:21.914623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99850566-2443-4cc0-9406-515a002dbc80 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:21.914820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99850566-2443-4cc0-9406-515a002dbc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:21.935035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-99850566-2443-4cc0-9406-515a002dbc80 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:21.936222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-99850566-2443-4cc0-9406-515a002dbc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:21.967737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:22.007065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 642/1294] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:26:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:26:22.033758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5e47b-d339-4404-a7e5-cc7105d028de None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:22.072289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:22.147159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5e47b-d339-4404-a7e5-cc7105d028de None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:22.183057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c65ef-04b0-4579-9f14-fa852f23d253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:22.183699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c65ef-04b0-4579-9f14-fa852f23d253 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:22.183982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c65ef-04b0-4579-9f14-fa852f23d253 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:22.184254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c65ef-04b0-4579-9f14-fa852f23d253 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:22.185003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c65ef-04b0-4579-9f14-fa852f23d253 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:22.186373 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c65ef-04b0-4579-9f14-fa852f23d253 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:22.201849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 653/1295] 10.222.0.69 () {60 vars in 1141 bytes} [Fri Jun 24 10:26:22 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:26:24.230492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bff25b-eb93-41f9-9186-b0ffbd0df10a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.231413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bff25b-eb93-41f9-9186-b0ffbd0df10a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.231413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bff25b-eb93-41f9-9186-b0ffbd0df10a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.233643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 643/1296] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:26:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:26:24.239380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf45d93-41fc-456e-bc84-551c57fbb5d3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.239793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf45d93-41fc-456e-bc84-551c57fbb5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.239959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf45d93-41fc-456e-bc84-551c57fbb5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.259267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-abf45d93-41fc-456e-bc84-551c57fbb5d3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:24.260446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-abf45d93-41fc-456e-bc84-551c57fbb5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:24.341555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 654/1297] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:26:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:26:24.365146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bff25b-eb93-41f9-9186-b0ffbd0df10a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.475771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bff25b-eb93-41f9-9186-b0ffbd0df10a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.510968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3460a-9196-4bdd-bde3-2b3c9178524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.511675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3460a-9196-4bdd-bde3-2b3c9178524e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.511958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3460a-9196-4bdd-bde3-2b3c9178524e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.512233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3460a-9196-4bdd-bde3-2b3c9178524e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.512932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3460a-9196-4bdd-bde3-2b3c9178524e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.514302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3460a-9196-4bdd-bde3-2b3c9178524e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-64e3460a-9196-4bdd-bde3-2b3c9178524e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:24.526344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:24.527536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:24.528424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 644/1298] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:26:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:24.534483 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf45d93-41fc-456e-bc84-551c57fbb5d3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.553540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:24.636416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf45d93-41fc-456e-bc84-551c57fbb5d3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.674035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f38ce4-05e7-4157-83d5-7b9d2e2ee2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.674694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f38ce4-05e7-4157-83d5-7b9d2e2ee2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.674970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f38ce4-05e7-4157-83d5-7b9d2e2ee2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.675254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f38ce4-05e7-4157-83d5-7b9d2e2ee2c5 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.675969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f38ce4-05e7-4157-83d5-7b9d2e2ee2c5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:24.676275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f38ce4-05e7-4157-83d5-7b9d2e2ee2c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.677699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f38ce4-05e7-4157-83d5-7b9d2e2ee2c5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.683188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 655/1299] 10.222.0.69 () {56 vars in 1116 bytes} [Fri Jun 24 10:26:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:24.689655 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3460a-9196-4bdd-bde3-2b3c9178524e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.722896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:24.793958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3460a-9196-4bdd-bde3-2b3c9178524e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.829115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c7671-47e9-4cf7-8227-3e737966745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.829765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7671-47e9-4cf7-8227-3e737966745b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.830050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7671-47e9-4cf7-8227-3e737966745b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.830328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7671-47e9-4cf7-8227-3e737966745b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.831122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c7671-47e9-4cf7-8227-3e737966745b None admin] RBAC: Adding query filter params (interface=public, service_id=8131fa56fe6d400487484aa9ed4a5cde, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:24.831442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c7671-47e9-4cf7-8227-3e737966745b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.832828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c7671-47e9-4cf7-8227-3e737966745b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.838426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 645/1300] 10.222.0.69 () {56 vars in 1251 bytes} [Fri Jun 24 10:26:24 2022] GET /identity/v3/endpoints?service_id=8131fa56fe6d400487484aa9ed4a5cde&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:24.844768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f38ce4-05e7-4157-83d5-7b9d2e2ee2c5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.927868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:24.947212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f38ce4-05e7-4157-83d5-7b9d2e2ee2c5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.983250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9664a9-26d7-402a-b931-9e1e205963a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.983973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9664a9-26d7-402a-b931-9e1e205963a1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.984263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9664a9-26d7-402a-b931-9e1e205963a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.984542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9664a9-26d7-402a-b931-9e1e205963a1 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.985262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9664a9-26d7-402a-b931-9e1e205963a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.986630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9664a9-26d7-402a-b931-9e1e205963a1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.993599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 656/1301] 10.222.0.69 () {56 vars in 1087 bytes} [Fri Jun 24 10:26:24 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:26:26.984326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb604616-2284-4980-9519-c85c709260ea None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:26.985272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb604616-2284-4980-9519-c85c709260ea None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:26.985272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb604616-2284-4980-9519-c85c709260ea None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:26.987485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 646/1302] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:26:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:26:26.992955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef0b5-5f9f-453b-8c04-a98c2642ff9d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:26.993389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef0b5-5f9f-453b-8c04-a98c2642ff9d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:26.993494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef0b5-5f9f-453b-8c04-a98c2642ff9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:27.012423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-698ef0b5-5f9f-453b-8c04-a98c2642ff9d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:27.013631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-698ef0b5-5f9f-453b-8c04-a98c2642ff9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:27.086288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 657/1303] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:26:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:26:27.109378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb604616-2284-4980-9519-c85c709260ea None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:27.219066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:27.227767 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb604616-2284-4980-9519-c85c709260ea None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:27.268706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398cb45-8d30-4043-9d46-b182f2a7d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:27.269602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398cb45-8d30-4043-9d46-b182f2a7d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:27.270002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398cb45-8d30-4043-9d46-b182f2a7d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:27.270303 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398cb45-8d30-4043-9d46-b182f2a7d6c0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:27.271139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398cb45-8d30-4043-9d46-b182f2a7d6c0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:27.272880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398cb45-8d30-4043-9d46-b182f2a7d6c0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-0398cb45-8d30-4043-9d46-b182f2a7d6c0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:26:27.286842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:26:27.287858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 24 10:26:27.288691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 647/1304] 10.222.0.69 () {56 vars in 1117 bytes} [Fri Jun 24 10:26:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:26:27.294291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef0b5-5f9f-453b-8c04-a98c2642ff9d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:27.374080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:27.402576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef0b5-5f9f-453b-8c04-a98c2642ff9d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:27.437895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542fd8d-970e-4604-badf-8b95bf3e6978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:27.438563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542fd8d-970e-4604-badf-8b95bf3e6978 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:27.438840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542fd8d-970e-4604-badf-8b95bf3e6978 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:27.439112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542fd8d-970e-4604-badf-8b95bf3e6978 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:27.439819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542fd8d-970e-4604-badf-8b95bf3e6978 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:27.440120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542fd8d-970e-4604-badf-8b95bf3e6978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:27.441537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542fd8d-970e-4604-badf-8b95bf3e6978 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:27.449909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 658/1305] 10.222.0.69 () {56 vars in 1116 bytes} [Fri Jun 24 10:26:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:27.455999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398cb45-8d30-4043-9d46-b182f2a7d6c0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:27.563172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398cb45-8d30-4043-9d46-b182f2a7d6c0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:27.598364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3f53-89d0-4bb3-a4eb-dca2a82cf5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:27.599007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3f53-89d0-4bb3-a4eb-dca2a82cf5e0 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:27.599292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3f53-89d0-4bb3-a4eb-dca2a82cf5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:27.599563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3f53-89d0-4bb3-a4eb-dca2a82cf5e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:27.600305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae3f53-89d0-4bb3-a4eb-dca2a82cf5e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:27.601714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae3f53-89d0-4bb3-a4eb-dca2a82cf5e0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:27.624195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 648/1306] 10.222.0.69 () {60 vars in 1144 bytes} [Fri Jun 24 10:26:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:26:37.444158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=30 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:37.619597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:30.961089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acb4e8-b0b3-4478-9d9e-126170cdf0ce None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:30.963015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acb4e8-b0b3-4478-9d9e-126170cdf0ce None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:30.963463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acb4e8-b0b3-4478-9d9e-126170cdf0ce None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:30.966417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 659/1307] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:30.971318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c902ef-0458-43cd-93a4-7c05bdbf4a07 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:30.972307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c902ef-0458-43cd-93a4-7c05bdbf4a07 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:30.972307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c902ef-0458-43cd-93a4-7c05bdbf4a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:31.157522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f6c902ef-0458-43cd-93a4-7c05bdbf4a07 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:31.159467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f6c902ef-0458-43cd-93a4-7c05bdbf4a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:31.365259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 649/1308] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:31.425859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5497a2-6649-4e24-8a07-fc32007863b6 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:31.426408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5497a2-6649-4e24-8a07-fc32007863b6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:31.426855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5497a2-6649-4e24-8a07-fc32007863b6 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:31.428399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 660/1309] 10.222.0.69 () {54 vars in 964 bytes} [Fri Jun 24 10:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:31.435789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841df53-f7da-4866-bd6f-8ea834c01826 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:31.436097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841df53-f7da-4866-bd6f-8ea834c01826 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:31.436374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841df53-f7da-4866-bd6f-8ea834c01826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:31.556085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7841df53-f7da-4866-bd6f-8ea834c01826 None None] MFA Rules not processed for user `a86de22b0e324c70aad792ec6b1a7e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:31.557608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7841df53-f7da-4866-bd6f-8ea834c01826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:31.588614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:31.659854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 650/1310] 10.222.0.69 () {58 vars in 1061 bytes} [Fri Jun 24 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:31.669610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5497a2-6649-4e24-8a07-fc32007863b6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:31.671858 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8d5497a2-6649-4e24-8a07-fc32007863b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:31.847635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5497a2-6649-4e24-8a07-fc32007863b6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:31.938618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf396221-3633-4294-a174-921a0815aa51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:31.939568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf396221-3633-4294-a174-921a0815aa51 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:31.940002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf396221-3633-4294-a174-921a0815aa51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:31.940581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf396221-3633-4294-a174-921a0815aa51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:31.942946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bf396221-3633-4294-a174-921a0815aa51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:32.009003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:32.075005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf396221-3633-4294-a174-921a0815aa51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:32.076478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf396221-3633-4294-a174-921a0815aa51 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:32.100875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 661/1311] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:34.461449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a18bb5-f635-48b3-8c5a-245b2be693f9 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:34.461913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a18bb5-f635-48b3-8c5a-245b2be693f9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:34.462004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a18bb5-f635-48b3-8c5a-245b2be693f9 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:34.463568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 651/1312] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:34.468970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc6368-031c-4f71-af0d-9d4917831b4a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:34.469403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc6368-031c-4f71-af0d-9d4917831b4a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:34.469768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc6368-031c-4f71-af0d-9d4917831b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:34.546936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3dbc6368-031c-4f71-af0d-9d4917831b4a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:34.548203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3dbc6368-031c-4f71-af0d-9d4917831b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:34.656999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 662/1313] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:34.710849 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3d280-d2a2-449b-a6f7-366eb0854a6a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:34.711068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3d280-d2a2-449b-a6f7-366eb0854a6a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:34.711299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3d280-d2a2-449b-a6f7-366eb0854a6a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:34.712700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 652/1314] 10.222.0.69 () {54 vars in 964 bytes} [Fri Jun 24 10:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:34.719352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c50cd-353e-4209-a413-0046f05f86d0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:34.719736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c50cd-353e-4209-a413-0046f05f86d0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:34.720088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c50cd-353e-4209-a413-0046f05f86d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:34.731594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:34.796882 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bd8c50cd-353e-4209-a413-0046f05f86d0 None None] MFA Rules not processed for user `a86de22b0e324c70aad792ec6b1a7e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:34.798084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bd8c50cd-353e-4209-a413-0046f05f86d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:34.876203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 663/1315] 10.222.0.69 () {58 vars in 1061 bytes} [Fri Jun 24 10:27:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:34.886094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3d280-d2a2-449b-a6f7-366eb0854a6a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:34.887653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a4a3d280-d2a2-449b-a6f7-366eb0854a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:34.998870 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3d280-d2a2-449b-a6f7-366eb0854a6a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:35.074071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:35.077004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6edfa1-4415-42e7-bd7e-f60efa3a334b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:35.077724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6edfa1-4415-42e7-bd7e-f60efa3a334b service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:35.078224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6edfa1-4415-42e7-bd7e-f60efa3a334b service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:35.078666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6edfa1-4415-42e7-bd7e-f60efa3a334b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:35.080581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5b6edfa1-4415-42e7-bd7e-f60efa3a334b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:35.194546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6edfa1-4415-42e7-bd7e-f60efa3a334b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:35.195992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6edfa1-4415-42e7-bd7e-f60efa3a334b service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:35.215694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 653/1316] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:37.752139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a7b5e-d244-4a17-9ef3-6e336b230140 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:37.752686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a7b5e-d244-4a17-9ef3-6e336b230140 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:37.752686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a7b5e-d244-4a17-9ef3-6e336b230140 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:37.754029 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 664/1317] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:37.759657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413687a-773d-46b0-b917-961213464735 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:37.760132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413687a-773d-46b0-b917-961213464735 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:37.760226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413687a-773d-46b0-b917-961213464735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:37.825279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e413687a-773d-46b0-b917-961213464735 None None] MFA Rules not processed for user `c60d4a5f5d83459c99e38bd82d610465`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:37.826717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e413687a-773d-46b0-b917-961213464735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:37.881012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:37.927052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 654/1318] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:27:37.970798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a7b5e-d244-4a17-9ef3-6e336b230140 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:37.973494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1d0a7b5e-d244-4a17-9ef3-6e336b230140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:38.103881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a7b5e-d244-4a17-9ef3-6e336b230140 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:38.106735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:38.178997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac8845-1fca-40f3-9a66-265cde9c12f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:38.179643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac8845-1fca-40f3-9a66-265cde9c12f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:38.179925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac8845-1fca-40f3-9a66-265cde9c12f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:38.180200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac8845-1fca-40f3-9a66-265cde9c12f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:38.181872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9dac8845-1fca-40f3-9a66-265cde9c12f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:38.294727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac8845-1fca-40f3-9a66-265cde9c12f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:38.296166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac8845-1fca-40f3-9a66-265cde9c12f3 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:38.315146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 665/1319] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:40.865492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4dbb3-abaf-469b-8816-fb5abc828670 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:40.866334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4dbb3-abaf-469b-8816-fb5abc828670 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:40.866334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4dbb3-abaf-469b-8816-fb5abc828670 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:40.868541 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 655/1320] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:40.877294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e331-3d40-48d5-9947-644eb10987dc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:40.877746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e331-3d40-48d5-9947-644eb10987dc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:40.877892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e331-3d40-48d5-9947-644eb10987dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:40.923862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:40.953118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3af5e331-3d40-48d5-9947-644eb10987dc None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:40.954319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3af5e331-3d40-48d5-9947-644eb10987dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:41.058194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 666/1321] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:41.104218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4dbb3-abaf-469b-8816-fb5abc828670 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:41.106819 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-44b4dbb3-abaf-469b-8816-fb5abc828670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:41.228256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4dbb3-abaf-469b-8816-fb5abc828670 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:41.309350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22658bb1-fc9e-4b70-8bfb-e4d4ccca0a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:41.309995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22658bb1-fc9e-4b70-8bfb-e4d4ccca0a4f service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:41.310270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22658bb1-fc9e-4b70-8bfb-e4d4ccca0a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:41.310537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22658bb1-fc9e-4b70-8bfb-e4d4ccca0a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:41.312168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-22658bb1-fc9e-4b70-8bfb-e4d4ccca0a4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:41.319270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:41.426814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22658bb1-fc9e-4b70-8bfb-e4d4ccca0a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:41.428234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22658bb1-fc9e-4b70-8bfb-e4d4ccca0a4f service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:41.446625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 656/1322] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:44.373612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9f654-0276-4a23-8407-71099582aa99 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:44.374199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9f654-0276-4a23-8407-71099582aa99 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:44.374199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9f654-0276-4a23-8407-71099582aa99 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:44.375508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 667/1323] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:44.380971 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5ba17-c465-494d-98ac-d6db2e717fff None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:44.381393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5ba17-c465-494d-98ac-d6db2e717fff None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:44.381676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5ba17-c465-494d-98ac-d6db2e717fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:44.450211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-32a5ba17-c465-494d-98ac-d6db2e717fff None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:44.451469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-32a5ba17-c465-494d-98ac-d6db2e717fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:44.517524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:44.562138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 657/1324] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:44.598810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9f654-0276-4a23-8407-71099582aa99 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:44.600392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-86d9f654-0276-4a23-8407-71099582aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:44.709780 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9f654-0276-4a23-8407-71099582aa99 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:44.723026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:44.786754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f323b0-14e2-4efe-952a-cca404b14a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:44.787409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f323b0-14e2-4efe-952a-cca404b14a13 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:44.787695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f323b0-14e2-4efe-952a-cca404b14a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:44.787976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f323b0-14e2-4efe-952a-cca404b14a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:44.789666 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-61f323b0-14e2-4efe-952a-cca404b14a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:44.904351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f323b0-14e2-4efe-952a-cca404b14a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:44.905733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f323b0-14e2-4efe-952a-cca404b14a13 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:44.924440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 668/1325] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:46.975874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83afd31-296e-4581-a709-0a17a7628f2f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:46.976409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83afd31-296e-4581-a709-0a17a7628f2f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:46.976409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83afd31-296e-4581-a709-0a17a7628f2f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:46.977798 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 658/1326] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:46.983223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e9c93-870d-4a78-b1f2-d2fbcc338eb4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:46.983640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e9c93-870d-4a78-b1f2-d2fbcc338eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:46.983821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e9c93-870d-4a78-b1f2-d2fbcc338eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:47.033683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:47.052648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-210e9c93-870d-4a78-b1f2-d2fbcc338eb4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:47.053881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-210e9c93-870d-4a78-b1f2-d2fbcc338eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:47.154252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 669/1327] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:47.189600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83afd31-296e-4581-a709-0a17a7628f2f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:47.191374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c83afd31-296e-4581-a709-0a17a7628f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:47.301829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83afd31-296e-4581-a709-0a17a7628f2f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:47.382709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42bf6d-08fc-43d3-8421-3495b437d833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:47.383359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42bf6d-08fc-43d3-8421-3495b437d833 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:47.383636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42bf6d-08fc-43d3-8421-3495b437d833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:47.383907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42bf6d-08fc-43d3-8421-3495b437d833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:47.385624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ef42bf6d-08fc-43d3-8421-3495b437d833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:47.404124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:47.508148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42bf6d-08fc-43d3-8421-3495b437d833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:47.509635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42bf6d-08fc-43d3-8421-3495b437d833 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:47.535043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 659/1328] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:49.586578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d610a8a-5ef6-4d60-a39a-5bdd50b36e12 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:49.587110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d610a8a-5ef6-4d60-a39a-5bdd50b36e12 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:49.587110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d610a8a-5ef6-4d60-a39a-5bdd50b36e12 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:49.588567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 670/1329] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:49.593906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec2d9e-ee62-479b-a319-482db0f92b3a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:49.594388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec2d9e-ee62-479b-a319-482db0f92b3a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:49.594503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec2d9e-ee62-479b-a319-482db0f92b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:49.664592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c4ec2d9e-ee62-479b-a319-482db0f92b3a None None] MFA Rules not processed for user `c60d4a5f5d83459c99e38bd82d610465`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:49.665967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c4ec2d9e-ee62-479b-a319-482db0f92b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:49.737621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:49.772107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 660/1330] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:49.808912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d610a8a-5ef6-4d60-a39a-5bdd50b36e12 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:49.810447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7d610a8a-5ef6-4d60-a39a-5bdd50b36e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:49.920073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d610a8a-5ef6-4d60-a39a-5bdd50b36e12 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:49.942101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:49.997145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707012d-7520-49f9-865e-648b4b2e4587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:49.997804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707012d-7520-49f9-865e-648b4b2e4587 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:49.998081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707012d-7520-49f9-865e-648b4b2e4587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:49.998349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707012d-7520-49f9-865e-648b4b2e4587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:49.999984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f707012d-7520-49f9-865e-648b4b2e4587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:50.131147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707012d-7520-49f9-865e-648b4b2e4587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:50.132622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707012d-7520-49f9-865e-648b4b2e4587 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:50.152898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 671/1331] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:49 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:52.456258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d690c0c-15c1-40f3-8d2b-b3754baf0020 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:52.456905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d690c0c-15c1-40f3-8d2b-b3754baf0020 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:52.456905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d690c0c-15c1-40f3-8d2b-b3754baf0020 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:52.458962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 661/1332] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:52.467137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37814eb-812d-466c-9a20-e6fd40c8e975 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:52.467623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37814eb-812d-466c-9a20-e6fd40c8e975 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:52.467717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37814eb-812d-466c-9a20-e6fd40c8e975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:52.531925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:52.535492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e37814eb-812d-466c-9a20-e6fd40c8e975 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:52.537722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e37814eb-812d-466c-9a20-e6fd40c8e975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:52.655258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 672/1333] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:27:52.691398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d690c0c-15c1-40f3-8d2b-b3754baf0020 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:52.693774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3d690c0c-15c1-40f3-8d2b-b3754baf0020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:52.809535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d690c0c-15c1-40f3-8d2b-b3754baf0020 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:52.886852 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfab527-7ea2-4c9f-9b5f-335d3091dbad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:52.887499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab527-7ea2-4c9f-9b5f-335d3091dbad service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:52.887774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab527-7ea2-4c9f-9b5f-335d3091dbad service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:52.888050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab527-7ea2-4c9f-9b5f-335d3091dbad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:52.889724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2cfab527-7ea2-4c9f-9b5f-335d3091dbad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:52.918638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:53.015342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfab527-7ea2-4c9f-9b5f-335d3091dbad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:53.016836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfab527-7ea2-4c9f-9b5f-335d3091dbad service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:53.036243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 662/1334] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:55.077231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be572e-1310-42ab-9097-180f870d2a10 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:55.077789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be572e-1310-42ab-9097-180f870d2a10 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:55.077789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be572e-1310-42ab-9097-180f870d2a10 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:55.079167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 673/1335] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:55.086160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffc854-7567-4450-9b4b-6ab9f79ca284 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:55.086843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffc854-7567-4450-9b4b-6ab9f79ca284 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:55.087075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffc854-7567-4450-9b4b-6ab9f79ca284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:55.166356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6bffc854-7567-4450-9b4b-6ab9f79ca284 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:55.167613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6bffc854-7567-4450-9b4b-6ab9f79ca284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:55.245380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:55.268717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 663/1336] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:55.311910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be572e-1310-42ab-9097-180f870d2a10 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:55.313553 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-69be572e-1310-42ab-9097-180f870d2a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:55.425519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be572e-1310-42ab-9097-180f870d2a10 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:55.454823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:55.503396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785f55d-8533-4da9-a9c6-6cdedafd72e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:55.504364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785f55d-8533-4da9-a9c6-6cdedafd72e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:55.504646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785f55d-8533-4da9-a9c6-6cdedafd72e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:55.504945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785f55d-8533-4da9-a9c6-6cdedafd72e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:55.506584 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3785f55d-8533-4da9-a9c6-6cdedafd72e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:55.623878 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785f55d-8533-4da9-a9c6-6cdedafd72e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:55.625278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785f55d-8533-4da9-a9c6-6cdedafd72e7 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:55.644455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 674/1337] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:58.070314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76153b2-0163-4d6c-9f0f-e8f63651674f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:58.070925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76153b2-0163-4d6c-9f0f-e8f63651674f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:58.070925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76153b2-0163-4d6c-9f0f-e8f63651674f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:58.073016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 664/1338] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:27:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:27:58.080631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9f79-b766-4820-9450-52fa8ed411d7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:58.081229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9f79-b766-4820-9450-52fa8ed411d7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:58.081344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9f79-b766-4820-9450-52fa8ed411d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:58.149105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e86b9f79-b766-4820-9450-52fa8ed411d7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:58.150316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e86b9f79-b766-4820-9450-52fa8ed411d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:58.158989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:58.254159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 675/1339] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:27:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:58.294090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76153b2-0163-4d6c-9f0f-e8f63651674f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:58.296555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d76153b2-0163-4d6c-9f0f-e8f63651674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:58.417975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76153b2-0163-4d6c-9f0f-e8f63651674f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:58.497430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea9107-91df-4700-8ecc-0aa2bb2bd3be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:58.498077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea9107-91df-4700-8ecc-0aa2bb2bd3be service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:58.498356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea9107-91df-4700-8ecc-0aa2bb2bd3be service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:58.498618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea9107-91df-4700-8ecc-0aa2bb2bd3be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:58.500247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8fea9107-91df-4700-8ecc-0aa2bb2bd3be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:58.536066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:58.619044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea9107-91df-4700-8ecc-0aa2bb2bd3be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:58.620517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea9107-91df-4700-8ecc-0aa2bb2bd3be service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:58.640067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 665/1340] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:02.707493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ecae5-c035-41b9-befa-a8947d65c746 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:02.708024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ecae5-c035-41b9-befa-a8947d65c746 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:02.708024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ecae5-c035-41b9-befa-a8947d65c746 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:02.709394 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 676/1341] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:28:02.715074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4f090-f736-4a9a-bcdc-b339a38b6efe None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:02.715535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4f090-f736-4a9a-bcdc-b339a38b6efe None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:02.715637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4f090-f736-4a9a-bcdc-b339a38b6efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:02.787942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7bf4f090-f736-4a9a-bcdc-b339a38b6efe None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:02.789207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7bf4f090-f736-4a9a-bcdc-b339a38b6efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:02.886472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:28:02.901228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 666/1342] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:02.939738 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ecae5-c035-41b9-befa-a8947d65c746 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:02.942217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d86ecae5-c035-41b9-befa-a8947d65c746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:03.043158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ecae5-c035-41b9-befa-a8947d65c746 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:03.047177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364fa53-eb50-41b4-92df-47eec3622823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:03.047822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364fa53-eb50-41b4-92df-47eec3622823 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:03.048103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364fa53-eb50-41b4-92df-47eec3622823 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:03.048375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364fa53-eb50-41b4-92df-47eec3622823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:03.057796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364fa53-eb50-41b4-92df-47eec3622823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:03.059234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364fa53-eb50-41b4-92df-47eec3622823 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:03.068925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 677/1343] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:05.799114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e296158-7a54-4ff0-ada4-8d2693c4d7c4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:05.799934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e296158-7a54-4ff0-ada4-8d2693c4d7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:05.799934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e296158-7a54-4ff0-ada4-8d2693c4d7c4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:05.801174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 667/1344] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:05.806596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eba26-a734-4cf6-90d8-d2f86e1baf21 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:05.807042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eba26-a734-4cf6-90d8-d2f86e1baf21 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:05.807200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eba26-a734-4cf6-90d8-d2f86e1baf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:05.838441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e49eba26-a734-4cf6-90d8-d2f86e1baf21 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:05.839699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e49eba26-a734-4cf6-90d8-d2f86e1baf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:05.872534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:28:05.927290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 678/1345] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:05.965916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e296158-7a54-4ff0-ada4-8d2693c4d7c4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:05.967917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4e296158-7a54-4ff0-ada4-8d2693c4d7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:05.981687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e296158-7a54-4ff0-ada4-8d2693c4d7c4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:05.986062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790905f-6509-4211-b072-e82a1e7cae10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:05.987052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790905f-6509-4211-b072-e82a1e7cae10 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:05.987382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790905f-6509-4211-b072-e82a1e7cae10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:05.987635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790905f-6509-4211-b072-e82a1e7cae10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:05.997863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790905f-6509-4211-b072-e82a1e7cae10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:05.999292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790905f-6509-4211-b072-e82a1e7cae10 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:06.008974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 668/1346] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:10.124681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cbb99-b2ed-432a-8b46-70856178803c None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:10.125256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cbb99-b2ed-432a-8b46-70856178803c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:10.125256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cbb99-b2ed-432a-8b46-70856178803c None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:10.126595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 679/1347] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:10.133900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb5888-320e-412d-a03a-ad4615085c09 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:10.134467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb5888-320e-412d-a03a-ad4615085c09 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:10.134590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb5888-320e-412d-a03a-ad4615085c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:10.156045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8edb5888-320e-412d-a03a-ad4615085c09 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:10.159436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8edb5888-320e-412d-a03a-ad4615085c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:10.172220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 669/1348] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:10.220194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cbb99-b2ed-432a-8b46-70856178803c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:10.233461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cbb99-b2ed-432a-8b46-70856178803c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:10.237065 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f38505-c0e8-4a09-9f71-bfcd1b56a7b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:10.237754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f38505-c0e8-4a09-9f71-bfcd1b56a7b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:10.238033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f38505-c0e8-4a09-9f71-bfcd1b56a7b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:10.238796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f38505-c0e8-4a09-9f71-bfcd1b56a7b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:10.251694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f38505-c0e8-4a09-9f71-bfcd1b56a7b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:10.253674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f38505-c0e8-4a09-9f71-bfcd1b56a7b5 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:10.266341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 680/1349] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:28:15.286459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c92758-37f9-47b3-90bc-91d22bbe8366 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:15.287418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c92758-37f9-47b3-90bc-91d22bbe8366 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:15.287897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c92758-37f9-47b3-90bc-91d22bbe8366 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:15.289946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 670/1350] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:28:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:15.295056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ac912-fb89-4857-a3a5-e3d87290e444 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:15.295497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ac912-fb89-4857-a3a5-e3d87290e444 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:15.295595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ac912-fb89-4857-a3a5-e3d87290e444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:15.376570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8c4ac912-fb89-4857-a3a5-e3d87290e444 None None] MFA Rules not processed for user `9373333a1fd64852a138b3f5913168c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:15.377861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8c4ac912-fb89-4857-a3a5-e3d87290e444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:15.452681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 681/1351] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:28:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:19.527053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a34bf-efe3-461f-9b82-7779882f3900 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:19.527606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a34bf-efe3-461f-9b82-7779882f3900 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:19.527606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a34bf-efe3-461f-9b82-7779882f3900 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:19.529217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 671/1352] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:19.534676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf0825-4c48-4296-bea0-fe69fe37d44a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:19.535511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf0825-4c48-4296-bea0-fe69fe37d44a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:19.536003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf0825-4c48-4296-bea0-fe69fe37d44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:19.557835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c0bf0825-4c48-4296-bea0-fe69fe37d44a None None] MFA Rules not processed for user `9373333a1fd64852a138b3f5913168c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:19.559549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c0bf0825-4c48-4296-bea0-fe69fe37d44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:19.570274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 682/1353] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:20.149645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=13 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:28:21.118798 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410422e8-1e1a-497c-90ff-ef61c8e6ab93 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:21.119473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410422e8-1e1a-497c-90ff-ef61c8e6ab93 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:21.119473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410422e8-1e1a-497c-90ff-ef61c8e6ab93 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:21.120967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 672/1354] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:21.125922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3264c6-985e-473c-8a11-20387392a729 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:21.126346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3264c6-985e-473c-8a11-20387392a729 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:21.126474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3264c6-985e-473c-8a11-20387392a729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:21.144857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-db3264c6-985e-473c-8a11-20387392a729 None None] MFA Rules not processed for user `9373333a1fd64852a138b3f5913168c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:21.146054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-db3264c6-985e-473c-8a11-20387392a729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:21.156109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 683/1355] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:28:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:24.824685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ab33c-0aac-4618-bc78-80c24e76a541 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:24.825284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ab33c-0aac-4618-bc78-80c24e76a541 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:24.825388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ab33c-0aac-4618-bc78-80c24e76a541 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:24.826980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 673/1356] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:24.832518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99bf3f-81d5-4094-bd7f-7f92f71cefaf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:24.833360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99bf3f-81d5-4094-bd7f-7f92f71cefaf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:24.833851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99bf3f-81d5-4094-bd7f-7f92f71cefaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:24.862963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ff99bf3f-81d5-4094-bd7f-7f92f71cefaf None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:24.864565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ff99bf3f-81d5-4094-bd7f-7f92f71cefaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:24.875849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 684/1357] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:24.930595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0defda8-35dc-495b-a884-6c7e7eabb05f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:24.930879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0defda8-35dc-495b-a884-6c7e7eabb05f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:24.931153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0defda8-35dc-495b-a884-6c7e7eabb05f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:24.932533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 674/1358] 10.222.0.69 () {54 vars in 961 bytes} [Fri Jun 24 10:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:24.940688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3abee6-1dda-4de9-bf67-6e58f48fc0f6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:24.941362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3abee6-1dda-4de9-bf67-6e58f48fc0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:24.942542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3abee6-1dda-4de9-bf67-6e58f48fc0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:24.995077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:28:25.005090 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6f3abee6-1dda-4de9-bf67-6e58f48fc0f6 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:25.006321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6f3abee6-1dda-4de9-bf67-6e58f48fc0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:25.099752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 685/1359] 10.222.0.69 () {58 vars in 1058 bytes} [Fri Jun 24 10:28:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:25.113332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0defda8-35dc-495b-a884-6c7e7eabb05f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:25.130705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0defda8-35dc-495b-a884-6c7e7eabb05f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:25.136401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66f03e-7453-4223-ad20-ce49579a1c56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:25.137343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66f03e-7453-4223-ad20-ce49579a1c56 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:25.137615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66f03e-7453-4223-ad20-ce49579a1c56 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:25.137890 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66f03e-7453-4223-ad20-ce49579a1c56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:25.147260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66f03e-7453-4223-ad20-ce49579a1c56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:25.148918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66f03e-7453-4223-ad20-ce49579a1c56 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:25.159396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 675/1360] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:27.373220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec3277-2792-4dde-8dee-97e9d8082491 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:27.373797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec3277-2792-4dde-8dee-97e9d8082491 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:27.373797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec3277-2792-4dde-8dee-97e9d8082491 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:27.375303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 686/1361] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:27.380653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb32bf9-fda3-4d42-937d-8a0602cd8a0c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:27.381103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb32bf9-fda3-4d42-937d-8a0602cd8a0c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:27.381318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb32bf9-fda3-4d42-937d-8a0602cd8a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:27.399990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8bb32bf9-fda3-4d42-937d-8a0602cd8a0c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:27.401334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8bb32bf9-fda3-4d42-937d-8a0602cd8a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:27.410047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 676/1362] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:27.454649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec3277-2792-4dde-8dee-97e9d8082491 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:27.470117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec3277-2792-4dde-8dee-97e9d8082491 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:27.475672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56205f-c5d8-4c9f-9af2-d7f790bcf140 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:27.476432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56205f-c5d8-4c9f-9af2-d7f790bcf140 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:27.476715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56205f-c5d8-4c9f-9af2-d7f790bcf140 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:27.477016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56205f-c5d8-4c9f-9af2-d7f790bcf140 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:27.485649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56205f-c5d8-4c9f-9af2-d7f790bcf140 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:27.487417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56205f-c5d8-4c9f-9af2-d7f790bcf140 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:27.498686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 687/1363] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:29.801984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b000674-1a2c-431d-8509-a5ff03d3306b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:29.802481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b000674-1a2c-431d-8509-a5ff03d3306b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:29.803039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b000674-1a2c-431d-8509-a5ff03d3306b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:29.804014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 677/1364] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:29.809959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce2c21-ca4f-4964-9b6d-576224a4d5a9 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:29.810435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce2c21-ca4f-4964-9b6d-576224a4d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:29.810533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce2c21-ca4f-4964-9b6d-576224a4d5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:29.829395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cdce2c21-ca4f-4964-9b6d-576224a4d5a9 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:29.830610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cdce2c21-ca4f-4964-9b6d-576224a4d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:29.839490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 688/1365] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:29.885358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13719a-4887-4fdf-9214-dea120475dbe None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:29.885642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13719a-4887-4fdf-9214-dea120475dbe None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:29.885917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13719a-4887-4fdf-9214-dea120475dbe None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:29.887304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 678/1366] 10.222.0.69 () {54 vars in 961 bytes} [Fri Jun 24 10:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:29.893494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1fd09-1e3b-48f2-8cb1-44b7b6116807 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:29.893926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1fd09-1e3b-48f2-8cb1-44b7b6116807 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:29.894259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1fd09-1e3b-48f2-8cb1-44b7b6116807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:29.917594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e3c1fd09-1e3b-48f2-8cb1-44b7b6116807 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:29.919014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e3c1fd09-1e3b-48f2-8cb1-44b7b6116807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:29.937027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 689/1367] 10.222.0.69 () {58 vars in 1058 bytes} [Fri Jun 24 10:28:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:29.938072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13719a-4887-4fdf-9214-dea120475dbe None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:29.953388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13719a-4887-4fdf-9214-dea120475dbe None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:29.959086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc81e2e-cd8d-4eba-83f4-be9c431e49a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:29.959736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc81e2e-cd8d-4eba-83f4-be9c431e49a7 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:29.960023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc81e2e-cd8d-4eba-83f4-be9c431e49a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:29.960308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc81e2e-cd8d-4eba-83f4-be9c431e49a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:29.969905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc81e2e-cd8d-4eba-83f4-be9c431e49a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:29.971588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc81e2e-cd8d-4eba-83f4-be9c431e49a7 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:29.981401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 679/1368] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:32.035548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d050d9d-1f78-4041-a35a-5dbd3e832b11 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:32.036096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d050d9d-1f78-4041-a35a-5dbd3e832b11 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:32.036624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d050d9d-1f78-4041-a35a-5dbd3e832b11 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:32.038531 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 690/1369] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:32.043570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a25a7-36fd-4c46-a690-ffb2351b1a4e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:32.044000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a25a7-36fd-4c46-a690-ffb2351b1a4e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:32.044337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a25a7-36fd-4c46-a690-ffb2351b1a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:32.063654 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b62a25a7-36fd-4c46-a690-ffb2351b1a4e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:32.065073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b62a25a7-36fd-4c46-a690-ffb2351b1a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:32.075208 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 680/1370] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:32.119442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d050d9d-1f78-4041-a35a-5dbd3e832b11 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:32.135541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d050d9d-1f78-4041-a35a-5dbd3e832b11 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:32.141242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb51b26-fb68-43a6-b422-cbd44197b0cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:32.141996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb51b26-fb68-43a6-b422-cbd44197b0cb service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:32.142352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb51b26-fb68-43a6-b422-cbd44197b0cb service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:32.142690 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb51b26-fb68-43a6-b422-cbd44197b0cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:32.151432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb51b26-fb68-43a6-b422-cbd44197b0cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:32.153131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb51b26-fb68-43a6-b422-cbd44197b0cb service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:32.163648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 691/1371] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:32.907537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460c82d-a1da-45ff-9c97-6bf1ba7d8af7 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:32.907813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460c82d-a1da-45ff-9c97-6bf1ba7d8af7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:32.908295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460c82d-a1da-45ff-9c97-6bf1ba7d8af7 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:32.909742 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 681/1372] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:32.915392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228350a-35af-44cf-bcc9-68464ad1c952 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:32.915648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228350a-35af-44cf-bcc9-68464ad1c952 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:32.916120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228350a-35af-44cf-bcc9-68464ad1c952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:32.934280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5228350a-35af-44cf-bcc9-68464ad1c952 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:32.935467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5228350a-35af-44cf-bcc9-68464ad1c952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:32.943826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 692/1373] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:33.255812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a72352-9f79-4d82-ba17-9d21d653fe5a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:33.256243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a72352-9f79-4d82-ba17-9d21d653fe5a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:33.256363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a72352-9f79-4d82-ba17-9d21d653fe5a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:33.257956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 682/1374] 10.222.0.69 () {54 vars in 963 bytes} [Fri Jun 24 10:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:33.263010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37450ad-e71c-4b68-b8b9-99db0d025de4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:33.263457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37450ad-e71c-4b68-b8b9-99db0d025de4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:33.263556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37450ad-e71c-4b68-b8b9-99db0d025de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:33.316837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b37450ad-e71c-4b68-b8b9-99db0d025de4 None None] MFA Rules not processed for user `5586ef8925ef4cc496d358283c21c962`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:33.318049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b37450ad-e71c-4b68-b8b9-99db0d025de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:33.381300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 693/1375] 10.222.0.69 () {58 vars in 1060 bytes} [Fri Jun 24 10:28:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:33.389080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a72352-9f79-4d82-ba17-9d21d653fe5a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:33.402391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a72352-9f79-4d82-ba17-9d21d653fe5a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:33.406711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a81087-ec14-4718-b555-d99cba708b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:33.407500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a81087-ec14-4718-b555-d99cba708b3b service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:33.407781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a81087-ec14-4718-b555-d99cba708b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:33.408217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a81087-ec14-4718-b555-d99cba708b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:33.420796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a81087-ec14-4718-b555-d99cba708b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:33.423619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a81087-ec14-4718-b555-d99cba708b3b service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:33.444989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 683/1376] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:34.224562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68686fb6-563e-4206-a87f-e135fd9fb986 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:34.224809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68686fb6-563e-4206-a87f-e135fd9fb986 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:34.225162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68686fb6-563e-4206-a87f-e135fd9fb986 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:34.229321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 694/1377] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:34.231590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5c051-96c2-4e9e-b792-996b3102a231 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:34.231862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5c051-96c2-4e9e-b792-996b3102a231 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:34.232178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5c051-96c2-4e9e-b792-996b3102a231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:34.250120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-25c5c051-96c2-4e9e-b792-996b3102a231 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:34.251359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-25c5c051-96c2-4e9e-b792-996b3102a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:34.261144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 684/1378] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:34.315084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68686fb6-563e-4206-a87f-e135fd9fb986 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:34.336189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68686fb6-563e-4206-a87f-e135fd9fb986 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:34.343063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4864423-1e26-4665-b87c-094b7263b827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:34.343786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4864423-1e26-4665-b87c-094b7263b827 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:34.344141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4864423-1e26-4665-b87c-094b7263b827 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:34.344489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4864423-1e26-4665-b87c-094b7263b827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:34.353552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4864423-1e26-4665-b87c-094b7263b827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:34.355151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4864423-1e26-4665-b87c-094b7263b827 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:34.358978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:28:34.366262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 695/1379] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:36.418172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925801-2480-40bd-885e-790760c98844 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:36.418704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925801-2480-40bd-885e-790760c98844 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:36.418834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925801-2480-40bd-885e-790760c98844 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:36.420248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 685/1380] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:36.425836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5337f-c405-45ba-92ad-7914337a98f8 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:36.426282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5337f-c405-45ba-92ad-7914337a98f8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:36.426375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5337f-c405-45ba-92ad-7914337a98f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:36.452268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-47c5337f-c405-45ba-92ad-7914337a98f8 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:36.453798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-47c5337f-c405-45ba-92ad-7914337a98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:36.464801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 696/1381] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:36.510710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8925801-2480-40bd-885e-790760c98844 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:36.526851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8925801-2480-40bd-885e-790760c98844 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:36.534756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a9f0d-55dc-4fe2-a8c2-f56c3c1cb9d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:36.535763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a9f0d-55dc-4fe2-a8c2-f56c3c1cb9d1 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:36.536046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a9f0d-55dc-4fe2-a8c2-f56c3c1cb9d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:36.536345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a9f0d-55dc-4fe2-a8c2-f56c3c1cb9d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:36.545615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a9f0d-55dc-4fe2-a8c2-f56c3c1cb9d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:36.547221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a9f0d-55dc-4fe2-a8c2-f56c3c1cb9d1 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:36.557475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 686/1382] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:38.613993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fcddc-caaa-42f3-ae6c-22b464e9f8d2 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:38.614494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fcddc-caaa-42f3-ae6c-22b464e9f8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:38.614494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fcddc-caaa-42f3-ae6c-22b464e9f8d2 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:38.615890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 697/1383] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:38.621530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bd1b8-76f1-41f7-9938-02e94e1c7307 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:38.621989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bd1b8-76f1-41f7-9938-02e94e1c7307 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:38.622082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bd1b8-76f1-41f7-9938-02e94e1c7307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:38.640187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-597bd1b8-76f1-41f7-9938-02e94e1c7307 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:38.641395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-597bd1b8-76f1-41f7-9938-02e94e1c7307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:38.650316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 687/1384] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:38.691749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fcddc-caaa-42f3-ae6c-22b464e9f8d2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:38.708122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fcddc-caaa-42f3-ae6c-22b464e9f8d2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:38.714395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a472fc-f809-4ad9-92b4-e302d046ab21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:38.716273 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a472fc-f809-4ad9-92b4-e302d046ab21 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:38.716661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a472fc-f809-4ad9-92b4-e302d046ab21 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:38.717062 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a472fc-f809-4ad9-92b4-e302d046ab21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:38.726167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a472fc-f809-4ad9-92b4-e302d046ab21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:38.727901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a472fc-f809-4ad9-92b4-e302d046ab21 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:38.738767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 698/1385] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:40.772924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cdf96-0f7c-45f0-9ef9-19dca05c38f6 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:40.772924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cdf96-0f7c-45f0-9ef9-19dca05c38f6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:40.772924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cdf96-0f7c-45f0-9ef9-19dca05c38f6 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:40.777824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25d234-cae2-4cf4-9453-574167a3f9c6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:40.778295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25d234-cae2-4cf4-9453-574167a3f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:40.778295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25d234-cae2-4cf4-9453-574167a3f9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:40.780898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 688/1386] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:40 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:40.797760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2a25d234-cae2-4cf4-9453-574167a3f9c6 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:40.798975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2a25d234-cae2-4cf4-9453-574167a3f9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:40.809300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 699/1387] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:40.858989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cdf96-0f7c-45f0-9ef9-19dca05c38f6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:40.877856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cdf96-0f7c-45f0-9ef9-19dca05c38f6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:40.889314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef07e9-8bd2-410d-9d0d-63999f2b7c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:40.889505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef07e9-8bd2-410d-9d0d-63999f2b7c4f service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:40.889767 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef07e9-8bd2-410d-9d0d-63999f2b7c4f service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:40.890722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef07e9-8bd2-410d-9d0d-63999f2b7c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:40.904223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef07e9-8bd2-410d-9d0d-63999f2b7c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:40.906399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef07e9-8bd2-410d-9d0d-63999f2b7c4f service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:40.916770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 689/1388] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:42.947809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc9714-5713-41de-a78b-0cb1470b9124 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:42.948377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc9714-5713-41de-a78b-0cb1470b9124 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:42.948473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc9714-5713-41de-a78b-0cb1470b9124 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:42.950465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 700/1389] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:28:42.956248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0bd7b-491b-4482-ad0a-0b2bcef7a3a4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:42.956680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0bd7b-491b-4482-ad0a-0b2bcef7a3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:42.956794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0bd7b-491b-4482-ad0a-0b2bcef7a3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:42.977807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6af0bd7b-491b-4482-ad0a-0b2bcef7a3a4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:42.979128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6af0bd7b-491b-4482-ad0a-0b2bcef7a3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:42.987228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 690/1390] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:43.042539 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc9714-5713-41de-a78b-0cb1470b9124 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:43.059521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc9714-5713-41de-a78b-0cb1470b9124 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:43.065052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548dfb8-3619-45a2-abd6-46e9a8fa663a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:43.065907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548dfb8-3619-45a2-abd6-46e9a8fa663a service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:43.066200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548dfb8-3619-45a2-abd6-46e9a8fa663a service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:43.066468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548dfb8-3619-45a2-abd6-46e9a8fa663a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:43.075589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548dfb8-3619-45a2-abd6-46e9a8fa663a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:43.077251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548dfb8-3619-45a2-abd6-46e9a8fa663a service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:43.088624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 701/1391] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:28:45.553036 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56099914-7027-4d28-9ea5-9f572bc0b1c8 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:45.553667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56099914-7027-4d28-9ea5-9f572bc0b1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:45.553667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56099914-7027-4d28-9ea5-9f572bc0b1c8 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:45.555229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 691/1392] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:45.561901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18aa159-4f63-4f1f-a874-edb92817aa27 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:45.562376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18aa159-4f63-4f1f-a874-edb92817aa27 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:45.562604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18aa159-4f63-4f1f-a874-edb92817aa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:45.584475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c18aa159-4f63-4f1f-a874-edb92817aa27 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:45.586006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c18aa159-4f63-4f1f-a874-edb92817aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:45.596186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 702/1393] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:45.651927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56099914-7027-4d28-9ea5-9f572bc0b1c8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:45.668045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56099914-7027-4d28-9ea5-9f572bc0b1c8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:45.673755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5700c7-6241-423b-beb4-54d2dc089220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:45.674398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5700c7-6241-423b-beb4-54d2dc089220 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:45.674675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5700c7-6241-423b-beb4-54d2dc089220 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:45.674996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5700c7-6241-423b-beb4-54d2dc089220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:45.683729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5700c7-6241-423b-beb4-54d2dc089220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:45.685364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5700c7-6241-423b-beb4-54d2dc089220 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:45.695401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 692/1394] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:47.743674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbdace-18b8-4c65-b1fd-65e58158b517 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:47.744215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbdace-18b8-4c65-b1fd-65e58158b517 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:47.744215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbdace-18b8-4c65-b1fd-65e58158b517 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:47.745580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 703/1395] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:47.751044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c418f-71be-47b3-a081-8e78ef7571e5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:47.751524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c418f-71be-47b3-a081-8e78ef7571e5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:47.751616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c418f-71be-47b3-a081-8e78ef7571e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:47.771762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-be3c418f-71be-47b3-a081-8e78ef7571e5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:47.773083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-be3c418f-71be-47b3-a081-8e78ef7571e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:47.782607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 693/1396] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:47.827384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbdace-18b8-4c65-b1fd-65e58158b517 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:47.843213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbdace-18b8-4c65-b1fd-65e58158b517 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:47.849075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc9f30-edae-4253-89aa-b193ecb9761c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:47.849697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc9f30-edae-4253-89aa-b193ecb9761c service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:47.850008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc9f30-edae-4253-89aa-b193ecb9761c service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:47.850286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc9f30-edae-4253-89aa-b193ecb9761c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:47.859373 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc9f30-edae-4253-89aa-b193ecb9761c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:47.861012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc9f30-edae-4253-89aa-b193ecb9761c service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:47.871344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 704/1397] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:48.684182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383622c-cb92-4434-aa38-141e9e72a57b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:48.684594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383622c-cb92-4434-aa38-141e9e72a57b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:48.684931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383622c-cb92-4434-aa38-141e9e72a57b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:48.686335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 694/1398] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:48.692041 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd800e6a-5b47-493c-9b4a-609e98791bde None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:48.692328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd800e6a-5b47-493c-9b4a-609e98791bde None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:48.692603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd800e6a-5b47-493c-9b4a-609e98791bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:48.711032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-dd800e6a-5b47-493c-9b4a-609e98791bde None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:48.712515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dd800e6a-5b47-493c-9b4a-609e98791bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:48.720729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 705/1399] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:48.775453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcf8bd-d582-4d56-acbc-48c0fcaa9694 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:48.775943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcf8bd-d582-4d56-acbc-48c0fcaa9694 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:48.776127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcf8bd-d582-4d56-acbc-48c0fcaa9694 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:48.777583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 695/1400] 10.222.0.69 () {54 vars in 962 bytes} [Fri Jun 24 10:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:48.782656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01955081-8763-4aa9-a2ac-a6285bef3f81 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:48.783131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01955081-8763-4aa9-a2ac-a6285bef3f81 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:48.783225 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01955081-8763-4aa9-a2ac-a6285bef3f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:48.833231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-01955081-8763-4aa9-a2ac-a6285bef3f81 None None] MFA Rules not processed for user `1b75a27126c9464989cec70e5104824c`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:48.834429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-01955081-8763-4aa9-a2ac-a6285bef3f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:48.897654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 706/1401] 10.222.0.69 () {58 vars in 1059 bytes} [Fri Jun 24 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:48.905114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcf8bd-d582-4d56-acbc-48c0fcaa9694 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:48.917287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcf8bd-d582-4d56-acbc-48c0fcaa9694 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:48.921439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b289c3e-7663-49a3-b836-724d252ad59b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b75a27126c9464989cec70e5104824c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:48.922081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289c3e-7663-49a3-b836-724d252ad59b service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:48.922362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289c3e-7663-49a3-b836-724d252ad59b service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:48.922636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289c3e-7663-49a3-b836-724d252ad59b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:48.932109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b289c3e-7663-49a3-b836-724d252ad59b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:48.933558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b289c3e-7663-49a3-b836-724d252ad59b service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:48.943246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 696/1402] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:28:49.658501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5098bc-fd7e-42cd-847e-501caa62344d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:49.658996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5098bc-fd7e-42cd-847e-501caa62344d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:49.659516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5098bc-fd7e-42cd-847e-501caa62344d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:49.710292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9a5098bc-fd7e-42cd-847e-501caa62344d None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:49.711866 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9a5098bc-fd7e-42cd-847e-501caa62344d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:49.751282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:28:49.777617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 707/1403] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:28:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:49.822387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53a086-395b-4537-a038-bb68c3a6622c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:49.822912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53a086-395b-4537-a038-bb68c3a6622c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:49.822912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53a086-395b-4537-a038-bb68c3a6622c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:49.824328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 697/1404] 10.222.0.69 () {54 vars in 942 bytes} [Fri Jun 24 10:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:49.830865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383761f-b446-45b8-8253-e6ef8b0583cd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:49.831989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383761f-b446-45b8-8253-e6ef8b0583cd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:49.831989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383761f-b446-45b8-8253-e6ef8b0583cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:49.881412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0383761f-b446-45b8-8253-e6ef8b0583cd None None] MFA Rules not processed for user `44d5bd3f9dfe45ecb8d1a51d248f3c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:49.882534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0383761f-b446-45b8-8253-e6ef8b0583cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:49.910903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d295857-a17f-4d0c-bfc0-7a6e8cdac71f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:49.911399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d295857-a17f-4d0c-bfc0-7a6e8cdac71f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:49.911675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d295857-a17f-4d0c-bfc0-7a6e8cdac71f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:49.913250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 698/1405] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:28:49.918018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e446f62-4839-4c8a-85cb-5e8474396a93 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:49.918302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e446f62-4839-4c8a-85cb-5e8474396a93 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:49.918570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e446f62-4839-4c8a-85cb-5e8474396a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:49.937103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7e446f62-4839-4c8a-85cb-5e8474396a93 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:28:49.938309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7e446f62-4839-4c8a-85cb-5e8474396a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:28:49.947114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 699/1406] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:49.958632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 708/1407] 10.222.0.69 () {58 vars in 1039 bytes} [Fri Jun 24 10:28:49 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:28:49.968233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e446f62-4839-4c8a-85cb-5e8474396a93 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:49.981177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e446f62-4839-4c8a-85cb-5e8474396a93 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:49.986598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434f27d-d840-4e92-bca4-95a8d36e3f23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:49.987316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434f27d-d840-4e92-bca4-95a8d36e3f23 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:49.987674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434f27d-d840-4e92-bca4-95a8d36e3f23 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:49.987952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434f27d-d840-4e92-bca4-95a8d36e3f23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:49.992969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383761f-b446-45b8-8253-e6ef8b0583cd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:28:49.997261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434f27d-d840-4e92-bca4-95a8d36e3f23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:49.998843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434f27d-d840-4e92-bca4-95a8d36e3f23 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:50.005128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383761f-b446-45b8-8253-e6ef8b0583cd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:28:50.007239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 700/1408] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:28:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:28:50.011346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d972eae-5a0d-42a7-a8a9-a281efd225db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:28:50.011981 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d972eae-5a0d-42a7-a8a9-a281efd225db service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:28:50.012224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d972eae-5a0d-42a7-a8a9-a281efd225db service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:28:50.012499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d972eae-5a0d-42a7-a8a9-a281efd225db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:28:50.028556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d972eae-5a0d-42a7-a8a9-a281efd225db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:28:50.030486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d972eae-5a0d-42a7-a8a9-a281efd225db service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:28:50.041991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 709/1409] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:00.004269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=69 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:29:00.036723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=59 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:29:11.797417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e701587-2eed-4e46-8311-2622f0f24806 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:11.798010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e701587-2eed-4e46-8311-2622f0f24806 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:11.798010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e701587-2eed-4e46-8311-2622f0f24806 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:11.799273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 701/1410] 10.222.0.69 () {54 vars in 918 bytes} [Fri Jun 24 10:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:11.805648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b85da-b668-4201-a072-a0063693c2ea None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:11.806131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b85da-b668-4201-a072-a0063693c2ea None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:11.806233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b85da-b668-4201-a072-a0063693c2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:11.829279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2a1b85da-b668-4201-a072-a0063693c2ea None None] MFA Rules not processed for user `9373333a1fd64852a138b3f5913168c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:11.830492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2a1b85da-b668-4201-a072-a0063693c2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:11.841040 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 710/1411] 10.222.0.69 () {58 vars in 1015 bytes} [Fri Jun 24 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:11.855548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e0543-c93a-4a64-879a-e69d396100aa None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:11.855834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e0543-c93a-4a64-879a-e69d396100aa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:11.856110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e0543-c93a-4a64-879a-e69d396100aa None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:11.857541 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 702/1412] 10.222.0.69 () {54 vars in 918 bytes} [Fri Jun 24 10:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:11.862349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed63bb-1faa-4f36-bd6d-5c3c6ebefb89 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:11.862617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed63bb-1faa-4f36-bd6d-5c3c6ebefb89 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:11.862927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed63bb-1faa-4f36-bd6d-5c3c6ebefb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:11.882750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cfed63bb-1faa-4f36-bd6d-5c3c6ebefb89 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:11.883928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cfed63bb-1faa-4f36-bd6d-5c3c6ebefb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:11.894054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 711/1413] 10.222.0.69 () {58 vars in 1015 bytes} [Fri Jun 24 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:14.386717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99701251-60db-41b5-b0db-9f4827dd90ed None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:14.387559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99701251-60db-41b5-b0db-9f4827dd90ed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:14.387559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99701251-60db-41b5-b0db-9f4827dd90ed None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:14.389693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:14.395269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4779b8-0e49-4081-9369-7a9c40698d02 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:14.395696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4779b8-0e49-4081-9369-7a9c40698d02 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:14.395811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4779b8-0e49-4081-9369-7a9c40698d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:14.415216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7e4779b8-0e49-4081-9369-7a9c40698d02 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:14.416412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7e4779b8-0e49-4081-9369-7a9c40698d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:14.426139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:14.469392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99701251-60db-41b5-b0db-9f4827dd90ed None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:14.484243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99701251-60db-41b5-b0db-9f4827dd90ed None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:14.490019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843ace0-0f28-417e-a851-a81f850c2a3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:14.490660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ace0-0f28-417e-a851-a81f850c2a3d service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:14.490942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ace0-0f28-417e-a851-a81f850c2a3d service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:14.491218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ace0-0f28-417e-a851-a81f850c2a3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:14.499478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843ace0-0f28-417e-a851-a81f850c2a3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:14.501084 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843ace0-0f28-417e-a851-a81f850c2a3d service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:14.510708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 704/1416] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:15.427422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf955-e256-4b03-9951-670ac4204559 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:15.428002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf955-e256-4b03-9951-670ac4204559 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:15.428002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf955-e256-4b03-9951-670ac4204559 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:15.429421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:15.435650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74080cb5-c2b6-402e-9214-92ead4572436 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:15.436263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74080cb5-c2b6-402e-9214-92ead4572436 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:15.436522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74080cb5-c2b6-402e-9214-92ead4572436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:15.457002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-74080cb5-c2b6-402e-9214-92ead4572436 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:15.458210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-74080cb5-c2b6-402e-9214-92ead4572436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:15.467236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:15.623958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf955-e256-4b03-9951-670ac4204559 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:15.643783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf955-e256-4b03-9951-670ac4204559 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:15.650780 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7260a49-38d7-472a-b042-e8e83ece9efd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:15.651416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7260a49-38d7-472a-b042-e8e83ece9efd service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:15.651699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7260a49-38d7-472a-b042-e8e83ece9efd service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:15.651974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7260a49-38d7-472a-b042-e8e83ece9efd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:15.661508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7260a49-38d7-472a-b042-e8e83ece9efd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:15.663119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7260a49-38d7-472a-b042-e8e83ece9efd service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:15.673079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 714/1419] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:16.549834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd2f01-fb25-44ca-9a56-eaf1cc7238bb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:16.550314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd2f01-fb25-44ca-9a56-eaf1cc7238bb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:16.550443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd2f01-fb25-44ca-9a56-eaf1cc7238bb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:16.551833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:16.556945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f356b34-59f7-44a1-b3e4-67f77d1c7008 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:16.557382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f356b34-59f7-44a1-b3e4-67f77d1c7008 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:16.557475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f356b34-59f7-44a1-b3e4-67f77d1c7008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:16.575928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0f356b34-59f7-44a1-b3e4-67f77d1c7008 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:16.577164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0f356b34-59f7-44a1-b3e4-67f77d1c7008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:16.585399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:16.740684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd2f01-fb25-44ca-9a56-eaf1cc7238bb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:16.765681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd2f01-fb25-44ca-9a56-eaf1cc7238bb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:16.773393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e1fde-5f19-46af-8946-f011c34ed540 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:16.774033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e1fde-5f19-46af-8946-f011c34ed540 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:16.774319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e1fde-5f19-46af-8946-f011c34ed540 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:16.774585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e1fde-5f19-46af-8946-f011c34ed540 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:16.784030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e1fde-5f19-46af-8946-f011c34ed540 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:16.785665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e1fde-5f19-46af-8946-f011c34ed540 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:16.796112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 707/1422] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:17.651404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848c0fe-e714-410c-995f-f1c8bf1026bd None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:17.651964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848c0fe-e714-410c-995f-f1c8bf1026bd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:17.651964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848c0fe-e714-410c-995f-f1c8bf1026bd None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:17.653390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:17.658781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458772c-3431-4dc8-bca9-f8a3e7f3d470 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:17.659229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458772c-3431-4dc8-bca9-f8a3e7f3d470 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:17.659488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458772c-3431-4dc8-bca9-f8a3e7f3d470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:17.679722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0458772c-3431-4dc8-bca9-f8a3e7f3d470 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:17.680970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0458772c-3431-4dc8-bca9-f8a3e7f3d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:17.690965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:17.845034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848c0fe-e714-410c-995f-f1c8bf1026bd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:17.870542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848c0fe-e714-410c-995f-f1c8bf1026bd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:17.880297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b261afb-b320-4257-a0a3-3f7e80be5417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:17.880964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b261afb-b320-4257-a0a3-3f7e80be5417 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:17.881246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b261afb-b320-4257-a0a3-3f7e80be5417 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:17.881521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b261afb-b320-4257-a0a3-3f7e80be5417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:17.890555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b261afb-b320-4257-a0a3-3f7e80be5417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:17.892147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b261afb-b320-4257-a0a3-3f7e80be5417 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:17.902264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 717/1425] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:18.792006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b7d24-7c1f-485b-82e9-5fbeaaabc256 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:18.792546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b7d24-7c1f-485b-82e9-5fbeaaabc256 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:18.792546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b7d24-7c1f-485b-82e9-5fbeaaabc256 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:18.793932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:18.799950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41f7c9-6ca0-48f2-93bd-f09fd9c93e5a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:18.800403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41f7c9-6ca0-48f2-93bd-f09fd9c93e5a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:18.800508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41f7c9-6ca0-48f2-93bd-f09fd9c93e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:18.821066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9d41f7c9-6ca0-48f2-93bd-f09fd9c93e5a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:18.822261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9d41f7c9-6ca0-48f2-93bd-f09fd9c93e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:18.832644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:18.995636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b7d24-7c1f-485b-82e9-5fbeaaabc256 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:19.020510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b7d24-7c1f-485b-82e9-5fbeaaabc256 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:19.026893 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1ca7a-ca6d-403f-8e2c-a9e6e5efb93b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:19.027530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1ca7a-ca6d-403f-8e2c-a9e6e5efb93b service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:19.027815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1ca7a-ca6d-403f-8e2c-a9e6e5efb93b service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:19.028090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1ca7a-ca6d-403f-8e2c-a9e6e5efb93b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:19.036372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1ca7a-ca6d-403f-8e2c-a9e6e5efb93b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:19.037984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1ca7a-ca6d-403f-8e2c-a9e6e5efb93b service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:19.047433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 710/1428] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:19.850489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216a769-be58-44e4-8b29-e52d7ccc8184 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:19.851369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216a769-be58-44e4-8b29-e52d7ccc8184 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:19.851369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216a769-be58-44e4-8b29-e52d7ccc8184 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:19.853599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:29:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:19.858670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba0bef-6f73-4b08-bb51-3f663af67f59 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:19.859091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba0bef-6f73-4b08-bb51-3f663af67f59 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:19.859202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba0bef-6f73-4b08-bb51-3f663af67f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:19.878578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6eba0bef-6f73-4b08-bb51-3f663af67f59 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:19.879762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6eba0bef-6f73-4b08-bb51-3f663af67f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:19.888077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:20.045707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216a769-be58-44e4-8b29-e52d7ccc8184 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:20.073115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216a769-be58-44e4-8b29-e52d7ccc8184 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:20.081366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00550f-cd40-4e72-b608-d77a88c4ee00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:20.082309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00550f-cd40-4e72-b608-d77a88c4ee00 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:20.082731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00550f-cd40-4e72-b608-d77a88c4ee00 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:20.083141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00550f-cd40-4e72-b608-d77a88c4ee00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:20.096059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00550f-cd40-4e72-b608-d77a88c4ee00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:20.097823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00550f-cd40-4e72-b608-d77a88c4ee00 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:20.109723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 720/1431] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:20.925271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672afe7-0748-464a-a921-0c1edc3cc612 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:20.925860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672afe7-0748-464a-a921-0c1edc3cc612 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:20.925860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672afe7-0748-464a-a921-0c1edc3cc612 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:20.927210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:20.933092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b5133-9b88-464e-ada0-ebbc0fbb2974 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:20.933821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b5133-9b88-464e-ada0-ebbc0fbb2974 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:20.933989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b5133-9b88-464e-ada0-ebbc0fbb2974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:20.959478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-dd4b5133-9b88-464e-ada0-ebbc0fbb2974 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:20.961872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dd4b5133-9b88-464e-ada0-ebbc0fbb2974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:20.975260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:21.127384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672afe7-0748-464a-a921-0c1edc3cc612 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:21.150540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672afe7-0748-464a-a921-0c1edc3cc612 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:21.157039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbe5fe-90c5-448b-9c29-bf937c01087d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:21.157685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe5fe-90c5-448b-9c29-bf937c01087d service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:21.157964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe5fe-90c5-448b-9c29-bf937c01087d service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:21.158264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe5fe-90c5-448b-9c29-bf937c01087d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:21.167913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe5fe-90c5-448b-9c29-bf937c01087d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:21.169552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe5fe-90c5-448b-9c29-bf937c01087d service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:21.179428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 713/1434] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:21.975145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c07897-adb2-4f44-aa5c-2d5e4f32a68e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:21.975729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c07897-adb2-4f44-aa5c-2d5e4f32a68e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:21.975729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c07897-adb2-4f44-aa5c-2d5e4f32a68e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:21.977286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:21.983392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569b892-8ed4-4c46-8157-2e0ca617d221 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:21.984060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569b892-8ed4-4c46-8157-2e0ca617d221 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:21.984207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569b892-8ed4-4c46-8157-2e0ca617d221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:22.004226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7569b892-8ed4-4c46-8157-2e0ca617d221 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:22.005464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7569b892-8ed4-4c46-8157-2e0ca617d221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:22.014372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:22.163022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c07897-adb2-4f44-aa5c-2d5e4f32a68e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:22.186899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c07897-adb2-4f44-aa5c-2d5e4f32a68e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:22.194833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc87e71-cb37-4b54-8a03-f23bd7d0fd50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:22.195473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc87e71-cb37-4b54-8a03-f23bd7d0fd50 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:22.195752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc87e71-cb37-4b54-8a03-f23bd7d0fd50 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:22.196022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc87e71-cb37-4b54-8a03-f23bd7d0fd50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:22.204772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc87e71-cb37-4b54-8a03-f23bd7d0fd50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:22.206435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc87e71-cb37-4b54-8a03-f23bd7d0fd50 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:22.216771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 723/1437] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:23.019377 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18551e82-4345-4977-8246-7a5dd4c809c4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:23.020280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18551e82-4345-4977-8246-7a5dd4c809c4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:23.020280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18551e82-4345-4977-8246-7a5dd4c809c4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:23.022471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:29:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:23.028715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8097fa-38f9-4bef-9637-f656ccaed36c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:23.029183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8097fa-38f9-4bef-9637-f656ccaed36c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:23.029307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8097fa-38f9-4bef-9637-f656ccaed36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:23.049801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3f8097fa-38f9-4bef-9637-f656ccaed36c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:23.050997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f8097fa-38f9-4bef-9637-f656ccaed36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:23.059411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:23.214680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18551e82-4345-4977-8246-7a5dd4c809c4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:23.232044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18551e82-4345-4977-8246-7a5dd4c809c4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:23.238139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c51bc7-3a34-4dfd-a44d-aa9b67b109d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:23.238771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51bc7-3a34-4dfd-a44d-aa9b67b109d0 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:23.239049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51bc7-3a34-4dfd-a44d-aa9b67b109d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:23.239319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51bc7-3a34-4dfd-a44d-aa9b67b109d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:23.247625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c51bc7-3a34-4dfd-a44d-aa9b67b109d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:23.249219 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c51bc7-3a34-4dfd-a44d-aa9b67b109d0 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:23.258804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 716/1440] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:25.178782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bbeaa-3814-4d82-9bf8-a95fd247d633 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:25.179410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bbeaa-3814-4d82-9bf8-a95fd247d633 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:25.179410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bbeaa-3814-4d82-9bf8-a95fd247d633 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:25.181560 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:25.187685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b6e5f-96c7-4c7c-a698-9cd05282ab5e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:25.188415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b6e5f-96c7-4c7c-a698-9cd05282ab5e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:25.188563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b6e5f-96c7-4c7c-a698-9cd05282ab5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:25.208362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2e7b6e5f-96c7-4c7c-a698-9cd05282ab5e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:25.209606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2e7b6e5f-96c7-4c7c-a698-9cd05282ab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:25.219242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:25.261331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bbeaa-3814-4d82-9bf8-a95fd247d633 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:25.274177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bbeaa-3814-4d82-9bf8-a95fd247d633 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:25.278948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a19ee4-e93c-47a6-8e18-9cb1f81b0075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:25.279854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19ee4-e93c-47a6-8e18-9cb1f81b0075 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:25.280203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19ee4-e93c-47a6-8e18-9cb1f81b0075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:25.280536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19ee4-e93c-47a6-8e18-9cb1f81b0075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:25.292005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a19ee4-e93c-47a6-8e18-9cb1f81b0075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:25.293480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a19ee4-e93c-47a6-8e18-9cb1f81b0075 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:25.303475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 726/1443] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:27.625992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f8736-71f7-49b1-9aab-94ee26cfd3ff None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:27.626577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f8736-71f7-49b1-9aab-94ee26cfd3ff None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:27.626577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f8736-71f7-49b1-9aab-94ee26cfd3ff None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:27.628056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:27.634596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f412b-237a-45e1-a873-16bb19678692 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:27.635296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f412b-237a-45e1-a873-16bb19678692 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:27.635443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f412b-237a-45e1-a873-16bb19678692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:27.656603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fe8f412b-237a-45e1-a873-16bb19678692 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:27.657827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fe8f412b-237a-45e1-a873-16bb19678692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:27.666427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:27.707505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f8736-71f7-49b1-9aab-94ee26cfd3ff None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:27.719904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f8736-71f7-49b1-9aab-94ee26cfd3ff None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:27.723887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21390d16-0d89-40d4-83f2-4d9fae0ce0f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:27.724521 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21390d16-0d89-40d4-83f2-4d9fae0ce0f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:27.724830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21390d16-0d89-40d4-83f2-4d9fae0ce0f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:27.725118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21390d16-0d89-40d4-83f2-4d9fae0ce0f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:27.734465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21390d16-0d89-40d4-83f2-4d9fae0ce0f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:27.735898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21390d16-0d89-40d4-83f2-4d9fae0ce0f0 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:27.745503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 719/1446] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:30.070684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230a71f-f360-4334-bbd1-def991188226 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:30.071692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230a71f-f360-4334-bbd1-def991188226 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:30.071692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230a71f-f360-4334-bbd1-def991188226 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:30.073750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:30.081929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b83f2-e250-4959-bf51-c72fc145c536 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:30.082381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b83f2-e250-4959-bf51-c72fc145c536 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:30.082492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b83f2-e250-4959-bf51-c72fc145c536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:30.101662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-169b83f2-e250-4959-bf51-c72fc145c536 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:30.102910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-169b83f2-e250-4959-bf51-c72fc145c536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:30.112409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:30.154689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230a71f-f360-4334-bbd1-def991188226 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:30.172367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230a71f-f360-4334-bbd1-def991188226 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:30.177471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cdc4a-19fa-4436-8be5-19e79fea6df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:30.178117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cdc4a-19fa-4436-8be5-19e79fea6df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:30.178374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cdc4a-19fa-4436-8be5-19e79fea6df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:30.178648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cdc4a-19fa-4436-8be5-19e79fea6df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:30.188546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cdc4a-19fa-4436-8be5-19e79fea6df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:30.190064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cdc4a-19fa-4436-8be5-19e79fea6df7 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:30.200072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 729/1449] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:32.316580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c9340-928e-4baf-8110-716901938c0d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:32.317560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c9340-928e-4baf-8110-716901938c0d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:32.317560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c9340-928e-4baf-8110-716901938c0d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:32.319815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:32.328060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1159f3-8d3e-4d76-9254-f5399dd493b5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:32.328565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1159f3-8d3e-4d76-9254-f5399dd493b5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:32.328565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1159f3-8d3e-4d76-9254-f5399dd493b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:32.355330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8e1159f3-8d3e-4d76-9254-f5399dd493b5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:32.357174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8e1159f3-8d3e-4d76-9254-f5399dd493b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:32.368783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:29:32.412209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c9340-928e-4baf-8110-716901938c0d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:32.429056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c9340-928e-4baf-8110-716901938c0d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:32.435416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f619b8d-f93b-4c4b-b9d3-a70882ddc0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:32.436305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f619b8d-f93b-4c4b-b9d3-a70882ddc0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:32.436660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f619b8d-f93b-4c4b-b9d3-a70882ddc0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:32.437052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f619b8d-f93b-4c4b-b9d3-a70882ddc0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:32.448594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f619b8d-f93b-4c4b-b9d3-a70882ddc0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:32.450086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f619b8d-f93b-4c4b-b9d3-a70882ddc0e7 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:32.460448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 722/1452] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:34.615774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c992fb3-b4a6-4357-8883-ee1d86d1bbbe None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:34.616631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c992fb3-b4a6-4357-8883-ee1d86d1bbbe None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:34.616631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c992fb3-b4a6-4357-8883-ee1d86d1bbbe None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:34.618936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:34.627706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51f6d-fa01-4ca3-b6ed-3a0eb58b7c39 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:34.628230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51f6d-fa01-4ca3-b6ed-3a0eb58b7c39 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:34.628345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51f6d-fa01-4ca3-b6ed-3a0eb58b7c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:34.649993 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c9b51f6d-fa01-4ca3-b6ed-3a0eb58b7c39 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:34.651189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c9b51f6d-fa01-4ca3-b6ed-3a0eb58b7c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:34.659535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:34.697559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c992fb3-b4a6-4357-8883-ee1d86d1bbbe None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:34.709556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c992fb3-b4a6-4357-8883-ee1d86d1bbbe None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:34.713802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290bae77-a8c6-4f9f-a436-706c401036bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:34.714440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bae77-a8c6-4f9f-a436-706c401036bf service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:34.714720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bae77-a8c6-4f9f-a436-706c401036bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:34.714991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bae77-a8c6-4f9f-a436-706c401036bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:34.725036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290bae77-a8c6-4f9f-a436-706c401036bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:34.726471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290bae77-a8c6-4f9f-a436-706c401036bf service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:34.736196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 732/1455] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:36.816810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a74d8-5d73-43af-9f0c-5cfb2e0298f4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:36.817646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a74d8-5d73-43af-9f0c-5cfb2e0298f4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:36.817646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a74d8-5d73-43af-9f0c-5cfb2e0298f4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:36.819936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:36.828661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb6286-8e3a-4e2c-b764-47ca62a5fa5a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:36.829123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb6286-8e3a-4e2c-b764-47ca62a5fa5a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:36.829300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb6286-8e3a-4e2c-b764-47ca62a5fa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:36.850631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3ffb6286-8e3a-4e2c-b764-47ca62a5fa5a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:36.851884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3ffb6286-8e3a-4e2c-b764-47ca62a5fa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:36.861249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:36.900385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a74d8-5d73-43af-9f0c-5cfb2e0298f4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:36.919593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a74d8-5d73-43af-9f0c-5cfb2e0298f4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:36.926437 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5c8a5-a6ec-4d35-b7f8-3ff3c2654abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:36.927330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5c8a5-a6ec-4d35-b7f8-3ff3c2654abf service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:36.927675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5c8a5-a6ec-4d35-b7f8-3ff3c2654abf service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:36.927953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5c8a5-a6ec-4d35-b7f8-3ff3c2654abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:36.938749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5c8a5-a6ec-4d35-b7f8-3ff3c2654abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:36.940363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5c8a5-a6ec-4d35-b7f8-3ff3c2654abf service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:36.950698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 725/1458] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:39.021937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2348e-138c-4a13-8260-1d5476a05587 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:39.022568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2348e-138c-4a13-8260-1d5476a05587 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:39.022568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2348e-138c-4a13-8260-1d5476a05587 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:39.023885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:39.030518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d888b6-50cb-4532-8b9b-96ba033f15c5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:39.031274 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d888b6-50cb-4532-8b9b-96ba033f15c5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:39.031391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d888b6-50cb-4532-8b9b-96ba033f15c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:39.051979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-37d888b6-50cb-4532-8b9b-96ba033f15c5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:39.053214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-37d888b6-50cb-4532-8b9b-96ba033f15c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:39.062199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:39.104890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2348e-138c-4a13-8260-1d5476a05587 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:39.116769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2348e-138c-4a13-8260-1d5476a05587 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:39.120409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cc826-2b4e-4145-bca6-12e8fa97e9f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:39.121076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cc826-2b4e-4145-bca6-12e8fa97e9f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:39.121354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cc826-2b4e-4145-bca6-12e8fa97e9f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:39.121625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cc826-2b4e-4145-bca6-12e8fa97e9f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:39.131016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cc826-2b4e-4145-bca6-12e8fa97e9f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:39.132448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cc826-2b4e-4145-bca6-12e8fa97e9f7 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:39.141809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 735/1461] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:41.222199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2a5b4-4a06-448a-bf40-131f9ff740c5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:41.222836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2a5b4-4a06-448a-bf40-131f9ff740c5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:41.222836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2a5b4-4a06-448a-bf40-131f9ff740c5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:41.224151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:41.230528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70674012-dea2-4b41-afe6-ca265190f845 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:41.231177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70674012-dea2-4b41-afe6-ca265190f845 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:41.231289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70674012-dea2-4b41-afe6-ca265190f845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:41.242151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=79 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:29:41.258679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-70674012-dea2-4b41-afe6-ca265190f845 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:41.259881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-70674012-dea2-4b41-afe6-ca265190f845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:41.268933 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:41.310183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2a5b4-4a06-448a-bf40-131f9ff740c5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:41.316905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=73 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:29:41.323898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2a5b4-4a06-448a-bf40-131f9ff740c5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:41.328149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a072a30-8036-4c8f-b7f6-4edcbe1b92f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:41.328840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a072a30-8036-4c8f-b7f6-4edcbe1b92f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:41.329151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a072a30-8036-4c8f-b7f6-4edcbe1b92f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:41.329435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a072a30-8036-4c8f-b7f6-4edcbe1b92f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:41.339189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a072a30-8036-4c8f-b7f6-4edcbe1b92f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:41.340624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a072a30-8036-4c8f-b7f6-4edcbe1b92f7 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:41.350571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 728/1464] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:43.411470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd0971-526f-4cc1-a4f9-a8624e5f0fbd None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:43.412052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd0971-526f-4cc1-a4f9-a8624e5f0fbd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:43.412052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd0971-526f-4cc1-a4f9-a8624e5f0fbd None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:43.413828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:43.421108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df89db-6fe5-4b50-9e51-0c198a86c129 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:43.421585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df89db-6fe5-4b50-9e51-0c198a86c129 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:43.421689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df89db-6fe5-4b50-9e51-0c198a86c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:43.441962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-31df89db-6fe5-4b50-9e51-0c198a86c129 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:43.443180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-31df89db-6fe5-4b50-9e51-0c198a86c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:43.451568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:43.488696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd0971-526f-4cc1-a4f9-a8624e5f0fbd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:43.500831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd0971-526f-4cc1-a4f9-a8624e5f0fbd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:43.504997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcccd2-32e1-46f4-831e-77e37fed77e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:43.505642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcccd2-32e1-46f4-831e-77e37fed77e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:43.505928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcccd2-32e1-46f4-831e-77e37fed77e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:43.506209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcccd2-32e1-46f4-831e-77e37fed77e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:43.515870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcccd2-32e1-46f4-831e-77e37fed77e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:43.517345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcccd2-32e1-46f4-831e-77e37fed77e3 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:43.527254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 738/1467] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:45.577420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d5b61-7a32-4eb0-9af2-0692a3209eaa None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:45.577976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d5b61-7a32-4eb0-9af2-0692a3209eaa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:45.577976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d5b61-7a32-4eb0-9af2-0692a3209eaa None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:45.579361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:29:45.585793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b2e16-8129-44ae-9340-f6a1834538da None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:45.586232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b2e16-8129-44ae-9340-f6a1834538da None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:45.586763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b2e16-8129-44ae-9340-f6a1834538da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:45.609845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d37b2e16-8129-44ae-9340-f6a1834538da None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:45.611055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d37b2e16-8129-44ae-9340-f6a1834538da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:45.621342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:45.658133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d5b61-7a32-4eb0-9af2-0692a3209eaa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:45.670345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d5b61-7a32-4eb0-9af2-0692a3209eaa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:45.675331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f9b3-8b44-4e5a-a08a-222ecc58c745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:45.675983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f9b3-8b44-4e5a-a08a-222ecc58c745 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:45.676262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f9b3-8b44-4e5a-a08a-222ecc58c745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:45.676534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f9b3-8b44-4e5a-a08a-222ecc58c745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:45.687589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3f9b3-8b44-4e5a-a08a-222ecc58c745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:45.689076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3f9b3-8b44-4e5a-a08a-222ecc58c745 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:45.700239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 731/1470] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:55.603848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=11 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:29:55.695042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=14 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:30:05.673628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88044d0-c7b5-4c2d-8ed0-76ae609d4765 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:05.674232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88044d0-c7b5-4c2d-8ed0-76ae609d4765 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:05.674232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88044d0-c7b5-4c2d-8ed0-76ae609d4765 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:05.676376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 740/1471] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:30:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:30:05.685001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b141a-e983-47ff-9533-b22d6f22b602 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:05.685469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b141a-e983-47ff-9533-b22d6f22b602 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:05.685571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b141a-e983-47ff-9533-b22d6f22b602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:05.704067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-046b141a-e983-47ff-9533-b22d6f22b602 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:30:05.705545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-046b141a-e983-47ff-9533-b22d6f22b602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:05.714349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 732/1472] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:30:05.762476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558ccea-8763-4c48-a6cf-fb111ef620de None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:05.762774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558ccea-8763-4c48-a6cf-fb111ef620de None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:05.763060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558ccea-8763-4c48-a6cf-fb111ef620de None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:05.764452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 741/1473] 10.222.0.69 () {54 vars in 962 bytes} [Fri Jun 24 10:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:30:05.770512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011c5b6-5c6a-4faa-8ef1-c2e6a57253eb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:05.770933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011c5b6-5c6a-4faa-8ef1-c2e6a57253eb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:05.771431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011c5b6-5c6a-4faa-8ef1-c2e6a57253eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:05.794939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4011c5b6-5c6a-4faa-8ef1-c2e6a57253eb None None] MFA Rules not processed for user `1b75a27126c9464989cec70e5104824c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:30:05.796140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4011c5b6-5c6a-4faa-8ef1-c2e6a57253eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:05.804210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 733/1474] 10.222.0.69 () {58 vars in 1059 bytes} [Fri Jun 24 10:30:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:30:05.816666 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558ccea-8763-4c48-a6cf-fb111ef620de None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:30:05.830616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558ccea-8763-4c48-a6cf-fb111ef620de None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:30:05.835104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229e1ea-33c0-4ad1-9be5-1e8bc7d4c527 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b75a27126c9464989cec70e5104824c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:30:05.835759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229e1ea-33c0-4ad1-9be5-1e8bc7d4c527 service glance] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:05.836041 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229e1ea-33c0-4ad1-9be5-1e8bc7d4c527 service glance] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:05.836321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229e1ea-33c0-4ad1-9be5-1e8bc7d4c527 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:05.847371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229e1ea-33c0-4ad1-9be5-1e8bc7d4c527 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:30:05.848847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229e1ea-33c0-4ad1-9be5-1e8bc7d4c527 service glance] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:30:05.858595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 742/1475] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:30:07.971385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00ada2-6dd2-433d-9fb2-de708be80c18 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:07.971959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00ada2-6dd2-433d-9fb2-de708be80c18 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:07.971959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00ada2-6dd2-433d-9fb2-de708be80c18 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:07.973407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 734/1476] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:30:07.978946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb729e1-51ad-4e11-a95e-ad5dace0bf1d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:07.979338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb729e1-51ad-4e11-a95e-ad5dace0bf1d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:07.979510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb729e1-51ad-4e11-a95e-ad5dace0bf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:07.998050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0eb729e1-51ad-4e11-a95e-ad5dace0bf1d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:30:07.999244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0eb729e1-51ad-4e11-a95e-ad5dace0bf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:08.007551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 743/1477] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:30:08.031592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00ada2-6dd2-433d-9fb2-de708be80c18 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:30:08.047380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00ada2-6dd2-433d-9fb2-de708be80c18 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:30:08.053799 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac3f1d-a33e-47a2-941d-1d43b93a41c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:30:08.054458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac3f1d-a33e-47a2-941d-1d43b93a41c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:08.054739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac3f1d-a33e-47a2-941d-1d43b93a41c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:08.055014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac3f1d-a33e-47a2-941d-1d43b93a41c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:08.055754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac3f1d-a33e-47a2-941d-1d43b93a41c4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:30:08.057596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-b9ac3f1d-a33e-47a2-941d-1d43b93a41c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:30:08.057596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 24 10:30:08.057596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: Jun 24 10:30:08.058750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac3f1d-a33e-47a2-941d-1d43b93a41c4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:30:08.078989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 735/1478] 10.222.0.69 () {56 vars in 1108 bytes} [Fri Jun 24 10:30:08 2022] GET /identity/v3/projects => generated 2553 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:30:10.109123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e89f3c-fdc6-456b-b7af-024a0afef60f None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:10.109701 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e89f3c-fdc6-456b-b7af-024a0afef60f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:10.109701 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e89f3c-fdc6-456b-b7af-024a0afef60f None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:10.111844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 744/1479] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:30:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:30:10.117631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c886c4-14a6-4fdd-b8c7-432a2b789ce3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:10.118082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c886c4-14a6-4fdd-b8c7-432a2b789ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:10.118174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c886c4-14a6-4fdd-b8c7-432a2b789ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:10.137651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-27c886c4-14a6-4fdd-b8c7-432a2b789ce3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:30:10.138829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-27c886c4-14a6-4fdd-b8c7-432a2b789ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:10.147231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 736/1480] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:30:10.190345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e89f3c-fdc6-456b-b7af-024a0afef60f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:30:10.206350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e89f3c-fdc6-456b-b7af-024a0afef60f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:30:10.212488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087842d-6620-4332-8d2a-3f596a3f4f12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:30:10.213199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087842d-6620-4332-8d2a-3f596a3f4f12 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:10.213485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087842d-6620-4332-8d2a-3f596a3f4f12 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:10.213765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087842d-6620-4332-8d2a-3f596a3f4f12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:10.222709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087842d-6620-4332-8d2a-3f596a3f4f12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:30:10.224428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087842d-6620-4332-8d2a-3f596a3f4f12 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:30:10.234544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 745/1481] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:30:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:30:12.623596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d41bd0-19c6-488a-82b8-2ff209c8354a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:12.624137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d41bd0-19c6-488a-82b8-2ff209c8354a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:12.624137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d41bd0-19c6-488a-82b8-2ff209c8354a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:12.625500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 737/1482] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:30:12.631063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76cde9-1ec7-483d-90ad-045ee4dbf07e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:12.631485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76cde9-1ec7-483d-90ad-045ee4dbf07e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:12.631618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76cde9-1ec7-483d-90ad-045ee4dbf07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:12.649506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1a76cde9-1ec7-483d-90ad-045ee4dbf07e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:30:12.650769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1a76cde9-1ec7-483d-90ad-045ee4dbf07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:12.659396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 746/1483] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:30:12.695549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d41bd0-19c6-488a-82b8-2ff209c8354a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:30:12.706564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d41bd0-19c6-488a-82b8-2ff209c8354a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:30:12.710293 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b641602-e400-4082-92f3-fe69bedbf313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:30:12.710873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b641602-e400-4082-92f3-fe69bedbf313 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:12.711150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b641602-e400-4082-92f3-fe69bedbf313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:12.711766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b641602-e400-4082-92f3-fe69bedbf313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:12.721160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b641602-e400-4082-92f3-fe69bedbf313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:30:12.722593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b641602-e400-4082-92f3-fe69bedbf313 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:30:12.732211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 738/1484] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:30:14.820783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4782c-920a-41f4-947a-340d58b4f939 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:14.821328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4782c-920a-41f4-947a-340d58b4f939 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:14.821328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4782c-920a-41f4-947a-340d58b4f939 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:14.822684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 747/1485] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:30:14.829053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f44edf-aa0f-4806-9596-057257e194c6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:14.830189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f44edf-aa0f-4806-9596-057257e194c6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:14.830689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f44edf-aa0f-4806-9596-057257e194c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:14.855762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-04f44edf-aa0f-4806-9596-057257e194c6 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:30:14.857017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-04f44edf-aa0f-4806-9596-057257e194c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:14.866412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 739/1486] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:30:14.908096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4782c-920a-41f4-947a-340d58b4f939 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:30:14.919757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4782c-920a-41f4-947a-340d58b4f939 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:30:14.924059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d04e8-bca4-4a3e-bf15-cc7a9c48d13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:30:14.924696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d04e8-bca4-4a3e-bf15-cc7a9c48d13a service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:14.925011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d04e8-bca4-4a3e-bf15-cc7a9c48d13a service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:14.925286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d04e8-bca4-4a3e-bf15-cc7a9c48d13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:14.935115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d04e8-bca4-4a3e-bf15-cc7a9c48d13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:30:14.936559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d04e8-bca4-4a3e-bf15-cc7a9c48d13a service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:30:14.946227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 748/1487] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:30:17.125130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2a2a6-1436-4c74-a9e9-87a684f30db8 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:17.125665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2a2a6-1436-4c74-a9e9-87a684f30db8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:17.125774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2a2a6-1436-4c74-a9e9-87a684f30db8 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:17.127384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 740/1488] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:30:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:30:17.133078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a4853-0ba4-410a-aaf7-806f2822bce8 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:17.133580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a4853-0ba4-410a-aaf7-806f2822bce8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:17.133690 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a4853-0ba4-410a-aaf7-806f2822bce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:17.151790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e99a4853-0ba4-410a-aaf7-806f2822bce8 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:30:17.153022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e99a4853-0ba4-410a-aaf7-806f2822bce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:17.161600 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 749/1489] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:30:17.210194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2a2a6-1436-4c74-a9e9-87a684f30db8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:30:17.226926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2a2a6-1436-4c74-a9e9-87a684f30db8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:30:17.231707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eef67-1e2b-4da5-abf2-34d4c970d7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:30:17.232358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eef67-1e2b-4da5-abf2-34d4c970d7ba service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:17.232640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eef67-1e2b-4da5-abf2-34d4c970d7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:17.232948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eef67-1e2b-4da5-abf2-34d4c970d7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:17.243706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799eef67-1e2b-4da5-abf2-34d4c970d7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:30:17.245178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799eef67-1e2b-4da5-abf2-34d4c970d7ba service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:30:17.255450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 741/1490] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:30:19.497288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f201bcd-2faa-4ae9-8c13-32c0200ba597 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:19.497822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f201bcd-2faa-4ae9-8c13-32c0200ba597 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:19.497822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f201bcd-2faa-4ae9-8c13-32c0200ba597 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:19.499954 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 750/1491] 10.222.0.69 () {54 vars in 926 bytes} [Fri Jun 24 10:30:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:30:19.505922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8f826-74e3-432d-a1ad-ea8d11a33e69 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:19.506345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8f826-74e3-432d-a1ad-ea8d11a33e69 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:19.506481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8f826-74e3-432d-a1ad-ea8d11a33e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:19.527135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-53b8f826-74e3-432d-a1ad-ea8d11a33e69 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:30:19.528358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-53b8f826-74e3-432d-a1ad-ea8d11a33e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:19.539046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 742/1492] 10.222.0.69 () {58 vars in 1023 bytes} [Fri Jun 24 10:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:30:19.579060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f201bcd-2faa-4ae9-8c13-32c0200ba597 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:30:19.591877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f201bcd-2faa-4ae9-8c13-32c0200ba597 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:30:19.596481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa17786-c73d-4fb5-89b1-73392915fcea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:30:19.597161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa17786-c73d-4fb5-89b1-73392915fcea service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:19.597447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa17786-c73d-4fb5-89b1-73392915fcea service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:19.597727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa17786-c73d-4fb5-89b1-73392915fcea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:19.607093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa17786-c73d-4fb5-89b1-73392915fcea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:30:19.608544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa17786-c73d-4fb5-89b1-73392915fcea service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:30:19.618170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 751/1493] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:30:29.519670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=33 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:30:29.613961 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=33 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:31:47.495042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456befb4-5f5b-4335-97f9-34e01d6b8cd4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:47.495602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456befb4-5f5b-4335-97f9-34e01d6b8cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:47.495602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456befb4-5f5b-4335-97f9-34e01d6b8cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:47.523142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-456befb4-5f5b-4335-97f9-34e01d6b8cd4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:31:47.524357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-456befb4-5f5b-4335-97f9-34e01d6b8cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:31:47.532657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 743/1494] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:31:47.542564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d179b-34a8-4ca6-ad94-d243f70d1f21 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:47.543066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d179b-34a8-4ca6-ad94-d243f70d1f21 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:47.543163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d179b-34a8-4ca6-ad94-d243f70d1f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:47.563064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-871d179b-34a8-4ca6-ad94-d243f70d1f21 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:31:47.564263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-871d179b-34a8-4ca6-ad94-d243f70d1f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:31:47.573387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 752/1495] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:31:47.578274 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456befb4-5f5b-4335-97f9-34e01d6b8cd4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:47.588837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456befb4-5f5b-4335-97f9-34e01d6b8cd4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:47.594549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:47.595198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:47.595476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:47.595745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:47.596480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:31:47.596798 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:47.597820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:31:47.597820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 24 10:31:47.597820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: Jun 24 10:31:47.598754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:47.612122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 744/1496] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:31:47.625009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d179b-34a8-4ca6-ad94-d243f70d1f21 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:47.637077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d179b-34a8-4ca6-ad94-d243f70d1f21 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:47.643271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961d932-002c-4af8-80f3-8b6c451cbf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:47.643966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961d932-002c-4af8-80f3-8b6c451cbf11 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:47.644250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961d932-002c-4af8-80f3-8b6c451cbf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:47.644535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961d932-002c-4af8-80f3-8b6c451cbf11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:47.645258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961d932-002c-4af8-80f3-8b6c451cbf11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:47.647118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-3961d932-002c-4af8-80f3-8b6c451cbf11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:31:47.647118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Jun 24 10:31:47.647118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: Jun 24 10:31:47.648337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961d932-002c-4af8-80f3-8b6c451cbf11 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:47.688966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 753/1497] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:31:47 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:31:47.693662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:47.704525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec34ca-572b-49a8-8c28-ffff37e98af3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:47.764770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce41f4-22c2-40b7-990f-71bd6c6a6c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:47.765391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce41f4-22c2-40b7-990f-71bd6c6a6c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:47.765704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce41f4-22c2-40b7-990f-71bd6c6a6c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:47.765976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce41f4-22c2-40b7-990f-71bd6c6a6c82 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:47.766826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce41f4-22c2-40b7-990f-71bd6c6a6c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:47.767842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-54ce41f4-22c2-40b7-990f-71bd6c6a6c82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:31:47.767842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 24 10:31:47.767842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: Jun 24 10:31:47.769063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce41f4-22c2-40b7-990f-71bd6c6a6c82 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:47.790595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 745/1498] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:31:47 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:31:47.794943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961d932-002c-4af8-80f3-8b6c451cbf11 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:47.805606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961d932-002c-4af8-80f3-8b6c451cbf11 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:47.811115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15324d29-c2d4-42a7-9c6f-e45b5ea3857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:47.811778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15324d29-c2d4-42a7-9c6f-e45b5ea3857c admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:47.812058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15324d29-c2d4-42a7-9c6f-e45b5ea3857c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:47.812333 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15324d29-c2d4-42a7-9c6f-e45b5ea3857c admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:47.813103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15324d29-c2d4-42a7-9c6f-e45b5ea3857c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:47.814150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-15324d29-c2d4-42a7-9c6f-e45b5ea3857c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:31:47.814150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Jun 24 10:31:47.814150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: Jun 24 10:31:47.815042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15324d29-c2d4-42a7-9c6f-e45b5ea3857c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:47.828213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 754/1499] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:31:47.832280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce41f4-22c2-40b7-990f-71bd6c6a6c82 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:47.843314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce41f4-22c2-40b7-990f-71bd6c6a6c82 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:47.848317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dba38-480a-4e44-baca-38065f907624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:47.849228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dba38-480a-4e44-baca-38065f907624 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:47.849514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dba38-480a-4e44-baca-38065f907624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:47.849793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dba38-480a-4e44-baca-38065f907624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:47.850524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dba38-480a-4e44-baca-38065f907624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:47.851543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-172dba38-480a-4e44-baca-38065f907624 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:31:47.851543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 24 10:31:47.851543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: Jun 24 10:31:47.852415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dba38-480a-4e44-baca-38065f907624 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:47.866086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 746/1500] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:31:47.870515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15324d29-c2d4-42a7-9c6f-e45b5ea3857c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:47.880828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15324d29-c2d4-42a7-9c6f-e45b5ea3857c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:47.886696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306c7cd-a46e-41c4-ba89-ad9b49574f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:47.887578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306c7cd-a46e-41c4-ba89-ad9b49574f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:47.887859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306c7cd-a46e-41c4-ba89-ad9b49574f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:47.888136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306c7cd-a46e-41c4-ba89-ad9b49574f0f admin admin] PATH_INFO: `/v3/projects/70b196ff18ad416ebbe97800a93701c5/users/e3dcb34cf18549a3aa11749b15bcdbd5/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:47.904408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306c7cd-a46e-41c4-ba89-ad9b49574f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70b196ff18ad416ebbe97800a93701c5, user_id=e3dcb34cf18549a3aa11749b15bcdbd5, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:47.905488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-b306c7cd-a46e-41c4-ba89-ad9b49574f0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:31:47.905488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Jun 24 10:31:47.905488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: Jun 24 10:31:47.907416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306c7cd-a46e-41c4-ba89-ad9b49574f0f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:47.922371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 755/1501] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:31:47 2022] PUT /identity/v3/projects/70b196ff18ad416ebbe97800a93701c5/users/e3dcb34cf18549a3aa11749b15bcdbd5/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:31:47.934112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534147c7-a659-4717-9a14-55a435a8cf9e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:47.934414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534147c7-a659-4717-9a14-55a435a8cf9e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:47.934678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534147c7-a659-4717-9a14-55a435a8cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:47.961286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-534147c7-a659-4717-9a14-55a435a8cf9e None None] MFA Rules not processed for user `e3dcb34cf18549a3aa11749b15bcdbd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:31:47.962815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-534147c7-a659-4717-9a14-55a435a8cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:31:48.034916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 747/1502] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:31:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:31:48.052894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d6859-bb0e-4ccd-91fd-9a6c6dc817d9 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:48.053228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d6859-bb0e-4ccd-91fd-9a6c6dc817d9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:48.053505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d6859-bb0e-4ccd-91fd-9a6c6dc817d9 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:48.054901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 756/1503] 10.222.0.69 () {54 vars in 963 bytes} [Fri Jun 24 10:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:31:48.060072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0427bd-9909-485c-b4c3-2e53934c5a0c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:48.060361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0427bd-9909-485c-b4c3-2e53934c5a0c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:48.060638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0427bd-9909-485c-b4c3-2e53934c5a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:48.083960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bc0427bd-9909-485c-b4c3-2e53934c5a0c None None] MFA Rules not processed for user `5586ef8925ef4cc496d358283c21c962`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:31:48.083960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bc0427bd-9909-485c-b4c3-2e53934c5a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:31:48.140109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:31:48.150498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 748/1504] 10.222.0.69 () {58 vars in 1060 bytes} [Fri Jun 24 10:31:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:31:48.158497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d6859-bb0e-4ccd-91fd-9a6c6dc817d9 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:48.170123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d6859-bb0e-4ccd-91fd-9a6c6dc817d9 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:48.174126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeeda5b-b6fc-449b-b2ab-d2b62c716eba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:48.174766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeeda5b-b6fc-449b-b2ab-d2b62c716eba service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:48.175054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeeda5b-b6fc-449b-b2ab-d2b62c716eba service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:48.175331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeeda5b-b6fc-449b-b2ab-d2b62c716eba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:48.184287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeeda5b-b6fc-449b-b2ab-d2b62c716eba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:48.185763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeeda5b-b6fc-449b-b2ab-d2b62c716eba service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:48.194873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 757/1505] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:31:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:31:48.238906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496eb77-612d-499e-b24a-1676d51a036e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:48.239198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496eb77-612d-499e-b24a-1676d51a036e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:48.239481 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496eb77-612d-499e-b24a-1676d51a036e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:48.240966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 749/1506] 10.222.0.69 () {52 vars in 829 bytes} [Fri Jun 24 10:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:31:48.244572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeeda5b-b6fc-449b-b2ab-d2b62c716eba service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:48.311520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeeda5b-b6fc-449b-b2ab-d2b62c716eba service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:48.317449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:48.318328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:48.318608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:48.318974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] PATH_INFO: `/v3/users/e3dcb34cf18549a3aa11749b15bcdbd5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:48.320201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3dcb34cf18549a3aa11749b15bcdbd5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:48.321261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:31:48.321261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Jun 24 10:31:48.321261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: Jun 24 10:31:48.322456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:48.394583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3dcb34cf18549a3aa11749b15bcdbd5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:48.413479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3dcb34cf18549a3aa11749b15bcdbd5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:48.424674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3dcb34cf18549a3aa11749b15bcdbd5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:48.425350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3dcb34cf18549a3aa11749b15bcdbd5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:48.434480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:31:48.439529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3dcb34cf18549a3aa11749b15bcdbd5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:48.447527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 758/1507] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:31:48 2022] DELETE /identity/v3/users/e3dcb34cf18549a3aa11749b15bcdbd5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:31:48.452071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc768d7-62de-46ce-ba98-c5f753852b83 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:48.452347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc768d7-62de-46ce-ba98-c5f753852b83 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:48.452684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc768d7-62de-46ce-ba98-c5f753852b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:48.471674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6bc768d7-62de-46ce-ba98-c5f753852b83 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:31:48.472910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6bc768d7-62de-46ce-ba98-c5f753852b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:31:48.535074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 750/1508] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:31:48.545399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:48.547621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:31:48.615397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de395d5-a2f9-4127-b519-34c7bc895bcb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:48.619871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1cbf6-1ea3-4919-941c-6b969cceaf16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:48.620525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1cbf6-1ea3-4919-941c-6b969cceaf16 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:48.620844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1cbf6-1ea3-4919-941c-6b969cceaf16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:48.621122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1cbf6-1ea3-4919-941c-6b969cceaf16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:48.631861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1cbf6-1ea3-4919-941c-6b969cceaf16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:48.633352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1cbf6-1ea3-4919-941c-6b969cceaf16 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:48.643030 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 759/1509] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:31:48.989472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc768d7-62de-46ce-ba98-c5f753852b83 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:31:48.991790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6bc768d7-62de-46ce-ba98-c5f753852b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:31:49.004512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc768d7-62de-46ce-ba98-c5f753852b83 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:31:49.010200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:31:49.011080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:31:49.011362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:31:49.011633 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] PATH_INFO: `/v3/projects/70b196ff18ad416ebbe97800a93701c5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:31:49.012917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70b196ff18ad416ebbe97800a93701c5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:31:49.013953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:31:49.013953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 24 10:31:49.013953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: Jun 24 10:31:49.015168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:31:49.077824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70b196ff18ad416ebbe97800a93701c5', 'request_id': 'req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:49.093261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70b196ff18ad416ebbe97800a93701c5', 'request_id': 'req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:49.105730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70b196ff18ad416ebbe97800a93701c5', 'request_id': 'req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:49.116806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] The token cache is being invalidate because project 70b196ff18ad416ebbe97800a93701c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:31:49.117114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:49.117810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-32dc5c2c-f35a-42d0-8b4b-70e7990a05b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:31:49.119416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 751/1510] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:31:48 2022] DELETE /identity/v3/projects/70b196ff18ad416ebbe97800a93701c5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:31:58.638672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=29 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:31:59.113809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:33:26.296132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bd19c-f7a0-4a8a-9f10-0f47ff2fa055 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:26.297334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bd19c-f7a0-4a8a-9f10-0f47ff2fa055 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:26.297334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bd19c-f7a0-4a8a-9f10-0f47ff2fa055 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:26.299583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:33:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:33:26.305099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48dd3b-fe0b-4e75-b65a-77b9606b7eaa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:26.305632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48dd3b-fe0b-4e75-b65a-77b9606b7eaa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:26.305743 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48dd3b-fe0b-4e75-b65a-77b9606b7eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:26.324380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0e48dd3b-fe0b-4e75-b65a-77b9606b7eaa None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:26.325642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0e48dd3b-fe0b-4e75-b65a-77b9606b7eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:26.387011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:33:26.541905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bd19c-f7a0-4a8a-9f10-0f47ff2fa055 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:33:26.544176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-335bd19c-f7a0-4a8a-9f10-0f47ff2fa055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:26.625016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bd19c-f7a0-4a8a-9f10-0f47ff2fa055 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:33:26.630515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af7a7e-43f0-44f0-9c2a-a4ad0be910e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:33:26.631165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7a7e-43f0-44f0-9c2a-a4ad0be910e3 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:26.631424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7a7e-43f0-44f0-9c2a-a4ad0be910e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:26.631698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7a7e-43f0-44f0-9c2a-a4ad0be910e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:26.639871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af7a7e-43f0-44f0-9c2a-a4ad0be910e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:33:26.641583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af7a7e-43f0-44f0-9c2a-a4ad0be910e3 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:33:26.651213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 761/1513] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:33:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:33:27.467597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717370b-e2e4-4deb-aab1-395210986a3d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:27.468398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717370b-e2e4-4deb-aab1-395210986a3d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:27.468563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717370b-e2e4-4deb-aab1-395210986a3d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:27.470877 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:33:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:33:27.476576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d76860-b0c8-47b3-9a26-1dadae13dabe None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:27.477062 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d76860-b0c8-47b3-9a26-1dadae13dabe None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:27.477254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d76860-b0c8-47b3-9a26-1dadae13dabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:27.502817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f5d76860-b0c8-47b3-9a26-1dadae13dabe None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:27.504048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f5d76860-b0c8-47b3-9a26-1dadae13dabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:27.513174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:33:27.670431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717370b-e2e4-4deb-aab1-395210986a3d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:33:27.694511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717370b-e2e4-4deb-aab1-395210986a3d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:33:27.702126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7f0a8-cf5b-462a-b04a-39125964e805 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:33:27.702797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7f0a8-cf5b-462a-b04a-39125964e805 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:27.703050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7f0a8-cf5b-462a-b04a-39125964e805 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:27.703359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7f0a8-cf5b-462a-b04a-39125964e805 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:27.716798 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7f0a8-cf5b-462a-b04a-39125964e805 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:33:27.718954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7f0a8-cf5b-462a-b04a-39125964e805 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:33:27.730081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 754/1516] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:33:28.381538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead1fe-1d5e-4418-b4c8-e23cf6fe2457 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:28.381783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead1fe-1d5e-4418-b4c8-e23cf6fe2457 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:28.382092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead1fe-1d5e-4418-b4c8-e23cf6fe2457 None None] PATH_INFO: `/v3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:28.383433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 763/1517] 10.222.0.20 () {54 vars in 925 bytes} [Fri Jun 24 10:33:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:33:28.389984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876bc45-0087-4dba-9cec-d32698a89e02 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:28.390293 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876bc45-0087-4dba-9cec-d32698a89e02 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:28.390542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876bc45-0087-4dba-9cec-d32698a89e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:28.410771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f876bc45-0087-4dba-9cec-d32698a89e02 None None] MFA Rules not processed for user `9373333a1fd64852a138b3f5913168c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:28.412332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f876bc45-0087-4dba-9cec-d32698a89e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:28.479186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 755/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Jun 24 10:33:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:33:32.289165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aa45f-d833-4de8-9da9-789beb68f261 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:32.289730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aa45f-d833-4de8-9da9-789beb68f261 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:32.289730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aa45f-d833-4de8-9da9-789beb68f261 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:32.291016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 764/1519] 10.222.0.69 () {54 vars in 960 bytes} [Fri Jun 24 10:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:33:32.296258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48cdf7-e84f-4f04-a31d-2e751a9d7000 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:32.296702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48cdf7-e84f-4f04-a31d-2e751a9d7000 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:32.296831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48cdf7-e84f-4f04-a31d-2e751a9d7000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:32.316087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8b48cdf7-e84f-4f04-a31d-2e751a9d7000 None None] MFA Rules not processed for user `9373333a1fd64852a138b3f5913168c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:32.317307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8b48cdf7-e84f-4f04-a31d-2e751a9d7000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:32.326414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 756/1520] 10.222.0.69 () {58 vars in 1057 bytes} [Fri Jun 24 10:33:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:33:32.333921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048aa45f-d833-4de8-9da9-789beb68f261 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:33:32.349294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048aa45f-d833-4de8-9da9-789beb68f261 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:33:32.355541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94484264-1459-4732-8594-505cc9f080c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:33:32.356164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94484264-1459-4732-8594-505cc9f080c7 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:32.356432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94484264-1459-4732-8594-505cc9f080c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:32.356703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94484264-1459-4732-8594-505cc9f080c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:32.368481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94484264-1459-4732-8594-505cc9f080c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:33:32.370212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94484264-1459-4732-8594-505cc9f080c7 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:33:32.379767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 765/1521] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:33:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:33:38.976439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f07a6-2106-42d1-b9b9-a577ca737cf4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:38.977082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f07a6-2106-42d1-b9b9-a577ca737cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:38.977082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f07a6-2106-42d1-b9b9-a577ca737cf4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:38.978437 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:33:38.984568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8004f-1ab6-4749-9603-bedd42fcd3b7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:38.985307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8004f-1ab6-4749-9603-bedd42fcd3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:38.985549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8004f-1ab6-4749-9603-bedd42fcd3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:39.007351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cbd8004f-1ab6-4749-9603-bedd42fcd3b7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:39.008945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cbd8004f-1ab6-4749-9603-bedd42fcd3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:39.017845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:33:39.179790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f07a6-2106-42d1-b9b9-a577ca737cf4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:33:39.196656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f07a6-2106-42d1-b9b9-a577ca737cf4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:33:39.203551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf0b09-5995-415b-9433-5153aa988c90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:33:39.204194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0b09-5995-415b-9433-5153aa988c90 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:39.204471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0b09-5995-415b-9433-5153aa988c90 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:39.204752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0b09-5995-415b-9433-5153aa988c90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:39.214019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf0b09-5995-415b-9433-5153aa988c90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:33:39.215621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf0b09-5995-415b-9433-5153aa988c90 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:33:39.225450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 758/1524] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:33:40.037773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7034d-c2c9-4dc0-bb58-9fa42ab2271e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:40.038777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7034d-c2c9-4dc0-bb58-9fa42ab2271e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:40.038777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7034d-c2c9-4dc0-bb58-9fa42ab2271e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:40.040147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:33:40.045413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0c206-244e-40e8-92d2-06e4500dd799 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:40.045854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0c206-244e-40e8-92d2-06e4500dd799 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:40.045955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0c206-244e-40e8-92d2-06e4500dd799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:40.066405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e1c0c206-244e-40e8-92d2-06e4500dd799 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:40.067576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e1c0c206-244e-40e8-92d2-06e4500dd799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:40.076051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:33:40.230040 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7034d-c2c9-4dc0-bb58-9fa42ab2271e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:33:40.255451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7034d-c2c9-4dc0-bb58-9fa42ab2271e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:33:40.263215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdd816-c440-4699-955f-1c70616269c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:33:40.263892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdd816-c440-4699-955f-1c70616269c8 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:40.264175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdd816-c440-4699-955f-1c70616269c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:40.264483 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdd816-c440-4699-955f-1c70616269c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:40.277447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdd816-c440-4699-955f-1c70616269c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:33:40.279900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdd816-c440-4699-955f-1c70616269c8 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:33:40.291681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 768/1527] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:33:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:33:44.254546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b803c-e585-4fce-a048-e57a93486942 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:44.255438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b803c-e585-4fce-a048-e57a93486942 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:44.255438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b803c-e585-4fce-a048-e57a93486942 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:44.257826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:33:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:33:44.264649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f700285-e324-44fd-bcf6-820f5d8c78ff None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:44.265391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f700285-e324-44fd-bcf6-820f5d8c78ff None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:44.265534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f700285-e324-44fd-bcf6-820f5d8c78ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:44.289266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5f700285-e324-44fd-bcf6-820f5d8c78ff None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:44.290489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5f700285-e324-44fd-bcf6-820f5d8c78ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:44.300369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:33:44.336791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b803c-e585-4fce-a048-e57a93486942 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:33:44.339166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-108b803c-e585-4fce-a048-e57a93486942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:44.412150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b803c-e585-4fce-a048-e57a93486942 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:33:44.416117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85658b19-0f5b-47c9-a545-9d1d9876226f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:33:44.416871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85658b19-0f5b-47c9-a545-9d1d9876226f service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:44.417161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85658b19-0f5b-47c9-a545-9d1d9876226f service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:44.417444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85658b19-0f5b-47c9-a545-9d1d9876226f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:44.427477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85658b19-0f5b-47c9-a545-9d1d9876226f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:33:44.428944 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85658b19-0f5b-47c9-a545-9d1d9876226f service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:33:44.434434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:33:44.439547 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 761/1530] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:33:46.462625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdd0ce-c6d8-4628-bf53-61fb828495bc None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:46.463669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdd0ce-c6d8-4628-bf53-61fb828495bc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:46.463811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdd0ce-c6d8-4628-bf53-61fb828495bc None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:46.465276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:33:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:33:46.470619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ace7-8053-4144-a53f-01e417a882ea None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:46.471079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ace7-8053-4144-a53f-01e417a882ea None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:46.471191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ace7-8053-4144-a53f-01e417a882ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:46.489689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3547ace7-8053-4144-a53f-01e417a882ea None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:46.490898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3547ace7-8053-4144-a53f-01e417a882ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:46.499583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:33:46.538719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdd0ce-c6d8-4628-bf53-61fb828495bc None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:33:46.557815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdd0ce-c6d8-4628-bf53-61fb828495bc None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:33:46.562318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2762b19-d099-42ae-bd55-303a34157b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:33:46.562956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2762b19-d099-42ae-bd55-303a34157b45 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:46.563235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2762b19-d099-42ae-bd55-303a34157b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:46.563505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2762b19-d099-42ae-bd55-303a34157b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:46.573556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2762b19-d099-42ae-bd55-303a34157b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:33:46.575120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2762b19-d099-42ae-bd55-303a34157b45 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:33:46.585946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 771/1533] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:33:50.136181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2466aa-1c92-4b29-bf32-1d8de66697b5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:50.137181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2466aa-1c92-4b29-bf32-1d8de66697b5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:50.137311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2466aa-1c92-4b29-bf32-1d8de66697b5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:50.139446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:33:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:33:50.145072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee93829-737e-49b8-8eaa-3d0472768290 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:50.145509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee93829-737e-49b8-8eaa-3d0472768290 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:50.145615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee93829-737e-49b8-8eaa-3d0472768290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:50.153367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=62 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:33:50.166024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fee93829-737e-49b8-8eaa-3d0472768290 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:50.167248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fee93829-737e-49b8-8eaa-3d0472768290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:50.175838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:33:50.490188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2466aa-1c92-4b29-bf32-1d8de66697b5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:33:50.492414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3c2466aa-1c92-4b29-bf32-1d8de66697b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:50.560308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2466aa-1c92-4b29-bf32-1d8de66697b5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:33:50.564580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a07791-88b3-4ec0-beac-70cbadd5887d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:33:50.565265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a07791-88b3-4ec0-beac-70cbadd5887d service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:50.565547 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a07791-88b3-4ec0-beac-70cbadd5887d service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:50.565846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a07791-88b3-4ec0-beac-70cbadd5887d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:50.575910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a07791-88b3-4ec0-beac-70cbadd5887d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:33:50.577386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a07791-88b3-4ec0-beac-70cbadd5887d service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:33:50.586935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 764/1536] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:33:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:33:52.641528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2624c7c-c0a5-4035-ac82-f4eb36406283 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:52.642096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2624c7c-c0a5-4035-ac82-f4eb36406283 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:52.642096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2624c7c-c0a5-4035-ac82-f4eb36406283 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:52.643487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:33:52.649124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc9a7c-cca5-4b5b-9388-3c128218c9e7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:52.649603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc9a7c-cca5-4b5b-9388-3c128218c9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:52.649703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc9a7c-cca5-4b5b-9388-3c128218c9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:52.669669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4fcc9a7c-cca5-4b5b-9388-3c128218c9e7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:33:52.670880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4fcc9a7c-cca5-4b5b-9388-3c128218c9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:52.679220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:33:52.981983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2624c7c-c0a5-4035-ac82-f4eb36406283 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:33:52.985268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c2624c7c-c0a5-4035-ac82-f4eb36406283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:33:53.008727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2624c7c-c0a5-4035-ac82-f4eb36406283 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:33:53.014075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52362d-4e4b-4277-8f8f-a2c63c0e65b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:33:53.014717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52362d-4e4b-4277-8f8f-a2c63c0e65b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:33:53.015000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52362d-4e4b-4277-8f8f-a2c63c0e65b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:33:53.015271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52362d-4e4b-4277-8f8f-a2c63c0e65b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:33:53.024983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52362d-4e4b-4277-8f8f-a2c63c0e65b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:33:53.026427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52362d-4e4b-4277-8f8f-a2c63c0e65b0 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:33:53.036061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 774/1539] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:34:02.664214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=32 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:34:03.031919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:20.011615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420651c-773d-4200-b34d-3fb9864327cb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.013310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420651c-773d-4200-b34d-3fb9864327cb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.013310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420651c-773d-4200-b34d-3fb9864327cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.035051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0420651c-773d-4200-b34d-3fb9864327cb None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.036286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0420651c-773d-4200-b34d-3fb9864327cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.047598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 766/1540] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.058356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a802392-d9ba-41ba-8e5f-f34a103f5467 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.058963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a802392-d9ba-41ba-8e5f-f34a103f5467 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.058963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a802392-d9ba-41ba-8e5f-f34a103f5467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.078741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8a802392-d9ba-41ba-8e5f-f34a103f5467 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.080285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8a802392-d9ba-41ba-8e5f-f34a103f5467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.094557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 775/1541] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.099466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420651c-773d-4200-b34d-3fb9864327cb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.114257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420651c-773d-4200-b34d-3fb9864327cb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.122015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7821da-3191-4f42-a3b4-8bd9cb1e34b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.122698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7821da-3191-4f42-a3b4-8bd9cb1e34b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.122978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7821da-3191-4f42-a3b4-8bd9cb1e34b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.123256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7821da-3191-4f42-a3b4-8bd9cb1e34b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.124003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7821da-3191-4f42-a3b4-8bd9cb1e34b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:20.124325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7821da-3191-4f42-a3b4-8bd9cb1e34b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.126028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7821da-3191-4f42-a3b4-8bd9cb1e34b3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.139385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 767/1542] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:20.147454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a802392-d9ba-41ba-8e5f-f34a103f5467 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.156931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a802392-d9ba-41ba-8e5f-f34a103f5467 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.164653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daed98-1af8-4c42-b116-07410bbe96dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.165450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daed98-1af8-4c42-b116-07410bbe96dd admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.165730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daed98-1af8-4c42-b116-07410bbe96dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.166005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daed98-1af8-4c42-b116-07410bbe96dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.167504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daed98-1af8-4c42-b116-07410bbe96dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.169497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535311b-be63-459e-afad-aa2d43635058 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.169803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535311b-be63-459e-afad-aa2d43635058 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.170088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535311b-be63-459e-afad-aa2d43635058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.182530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daed98-1af8-4c42-b116-07410bbe96dd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.209579 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1535311b-be63-459e-afad-aa2d43635058 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.210832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1535311b-be63-459e-afad-aa2d43635058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.231471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 768/1543] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:35:20.233000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cc102-f7dc-49db-9558-4b2d36c2739d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.233324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cc102-f7dc-49db-9558-4b2d36c2739d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.233609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cc102-f7dc-49db-9558-4b2d36c2739d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.238730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 776/1544] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:20.240289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8d6f4-dce2-4b0a-aa81-b703c440a595 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.240591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8d6f4-dce2-4b0a-aa81-b703c440a595 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.245155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8d6f4-dce2-4b0a-aa81-b703c440a595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.278088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-dad8d6f4-dce2-4b0a-aa81-b703c440a595 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.278469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dad8d6f4-dce2-4b0a-aa81-b703c440a595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.282124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-555cc102-f7dc-49db-9558-4b2d36c2739d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.283631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-555cc102-f7dc-49db-9558-4b2d36c2739d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.357423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 777/1545] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.362291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8d6f4-dce2-4b0a-aa81-b703c440a595 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.369678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8d6f4-dce2-4b0a-aa81-b703c440a595 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.375793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e3962-7829-4dd8-b7f3-e3aed76efabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.376468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e3962-7829-4dd8-b7f3-e3aed76efabf admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.376763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e3962-7829-4dd8-b7f3-e3aed76efabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.377225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 769/1546] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.377435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e3962-7829-4dd8-b7f3-e3aed76efabf admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.378121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e3962-7829-4dd8-b7f3-e3aed76efabf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.379161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-fe4e3962-7829-4dd8-b7f3-e3aed76efabf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:35:20.379161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Jun 24 10:35:20.379161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: Jun 24 10:35:20.379966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4008d1-3091-4612-b63d-e10427278f86 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.380464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4008d1-3091-4612-b63d-e10427278f86 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.380578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e3962-7829-4dd8-b7f3-e3aed76efabf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.381257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4008d1-3091-4612-b63d-e10427278f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.402119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7f4008d1-3091-4612-b63d-e10427278f86 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.404510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7f4008d1-3091-4612-b63d-e10427278f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.412247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 778/1547] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/users => generated 288 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:20.413381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 770/1548] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.413715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799a749-75b2-4c9b-8d58-36f0e7de3024 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.414007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799a749-75b2-4c9b-8d58-36f0e7de3024 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.414775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880174a6-d5c1-4a3d-9927-db9a67fadd21 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.415058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880174a6-d5c1-4a3d-9927-db9a67fadd21 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.415338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880174a6-d5c1-4a3d-9927-db9a67fadd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.417272 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799a749-75b2-4c9b-8d58-36f0e7de3024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.433838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-880174a6-d5c1-4a3d-9927-db9a67fadd21 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.435185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-880174a6-d5c1-4a3d-9927-db9a67fadd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.435372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8799a749-75b2-4c9b-8d58-36f0e7de3024 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.436433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8799a749-75b2-4c9b-8d58-36f0e7de3024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.443125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 771/1549] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.444374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 779/1550] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.444546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274990c-31bb-47c9-8e77-161a3ee89bb0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.451780 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274990c-31bb-47c9-8e77-161a3ee89bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.451780 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274990c-31bb-47c9-8e77-161a3ee89bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.452039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c7c06-66d8-41da-8638-635718355fea None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.452039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c7c06-66d8-41da-8638-635718355fea None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.452039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c7c06-66d8-41da-8638-635718355fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.465329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d45c7c06-66d8-41da-8638-635718355fea None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.466524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d45c7c06-66d8-41da-8638-635718355fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.469362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e274990c-31bb-47c9-8e77-161a3ee89bb0 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.470596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e274990c-31bb-47c9-8e77-161a3ee89bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.474267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 780/1551] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.475478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072150c7-59fd-41fe-8700-fd6dc79e1518 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.475764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072150c7-59fd-41fe-8700-fd6dc79e1518 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.476037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072150c7-59fd-41fe-8700-fd6dc79e1518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.481939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 772/1552] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.483089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3700c34-f97d-4d76-8426-ebeb5ab8744b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.483477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3700c34-f97d-4d76-8426-ebeb5ab8744b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.483832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3700c34-f97d-4d76-8426-ebeb5ab8744b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.493964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-072150c7-59fd-41fe-8700-fd6dc79e1518 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.495142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-072150c7-59fd-41fe-8700-fd6dc79e1518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.501478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e3700c34-f97d-4d76-8426-ebeb5ab8744b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.502652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e3700c34-f97d-4d76-8426-ebeb5ab8744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.503498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 781/1553] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.504346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072150c7-59fd-41fe-8700-fd6dc79e1518 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.510532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 773/1554] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.511659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe66fba-82f2-4ac3-ad45-b7e47006ba65 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.511948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe66fba-82f2-4ac3-ad45-b7e47006ba65 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.512214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe66fba-82f2-4ac3-ad45-b7e47006ba65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.518045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072150c7-59fd-41fe-8700-fd6dc79e1518 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.523650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2abc-938e-4502-b129-1e1f1eeb15f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.524322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2abc-938e-4502-b129-1e1f1eeb15f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.525048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2abc-938e-4502-b129-1e1f1eeb15f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.525328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2abc-938e-4502-b129-1e1f1eeb15f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.526068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2abc-938e-4502-b129-1e1f1eeb15f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.527689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2abc-938e-4502-b129-1e1f1eeb15f0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.530296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0fe66fba-82f2-4ac3-ad45-b7e47006ba65 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.531438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0fe66fba-82f2-4ac3-ad45-b7e47006ba65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.540874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 782/1555] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:20.540874 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2abc-938e-4502-b129-1e1f1eeb15f0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.541119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 774/1556] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.541212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc8b94-d6fd-4042-9941-a33cd03f14f1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.541737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc8b94-d6fd-4042-9941-a33cd03f14f1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.541737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc8b94-d6fd-4042-9941-a33cd03f14f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.548033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2abc-938e-4502-b129-1e1f1eeb15f0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.552622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.553297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.553577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.553851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.554568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:20.554887 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.555947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:35:20.555947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Jun 24 10:35:20.555947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: Jun 24 10:35:20.556853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.559450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a0bc8b94-d6fd-4042-9941-a33cd03f14f1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.560642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a0bc8b94-d6fd-4042-9941-a33cd03f14f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.569186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 783/1557] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:20.569825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 775/1558] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.569990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.574031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc8b94-d6fd-4042-9941-a33cd03f14f1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.579753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27ac1e-5930-4fed-b6f7-58598eba76bb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.583602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc8b94-d6fd-4042-9941-a33cd03f14f1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.584355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9967f2-8ea0-4179-acc6-8f11443c93a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.585028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9967f2-8ea0-4179-acc6-8f11443c93a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.585310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9967f2-8ea0-4179-acc6-8f11443c93a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.585610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9967f2-8ea0-4179-acc6-8f11443c93a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.586327 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9967f2-8ea0-4179-acc6-8f11443c93a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:20.586624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9967f2-8ea0-4179-acc6-8f11443c93a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.588249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9967f2-8ea0-4179-acc6-8f11443c93a5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.588714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce094e20-6355-4700-aff2-1e3c1a782d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.589712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094e20-6355-4700-aff2-1e3c1a782d87 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.589993 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094e20-6355-4700-aff2-1e3c1a782d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.590264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094e20-6355-4700-aff2-1e3c1a782d87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.590992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce094e20-6355-4700-aff2-1e3c1a782d87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:20.591328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce094e20-6355-4700-aff2-1e3c1a782d87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.592972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce094e20-6355-4700-aff2-1e3c1a782d87 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.600379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 784/1559] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:20.601624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963585e-91b0-4c51-8e85-44ef197feb49 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.601912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963585e-91b0-4c51-8e85-44ef197feb49 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.602188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963585e-91b0-4c51-8e85-44ef197feb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.612991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 776/1560] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:20.614199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ea763-58b8-4714-81b7-90cf6f86326c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.614490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ea763-58b8-4714-81b7-90cf6f86326c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.614771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ea763-58b8-4714-81b7-90cf6f86326c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.625536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4963585e-91b0-4c51-8e85-44ef197feb49 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.626839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4963585e-91b0-4c51-8e85-44ef197feb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.633436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-221ea763-58b8-4714-81b7-90cf6f86326c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:20.634795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-221ea763-58b8-4714-81b7-90cf6f86326c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:20.636065 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 785/1561] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.637578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963585e-91b0-4c51-8e85-44ef197feb49 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.644422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 777/1562] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:20.647298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963585e-91b0-4c51-8e85-44ef197feb49 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.649051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ea763-58b8-4714-81b7-90cf6f86326c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.652066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a79de-c5e8-40ae-b2b6-2ec0673a9d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.652725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a79de-c5e8-40ae-b2b6-2ec0673a9d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.653025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a79de-c5e8-40ae-b2b6-2ec0673a9d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.653328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a79de-c5e8-40ae-b2b6-2ec0673a9d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.654086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a79de-c5e8-40ae-b2b6-2ec0673a9d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.655722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a79de-c5e8-40ae-b2b6-2ec0673a9d20 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.659317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ea763-58b8-4714-81b7-90cf6f86326c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.664218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373287-c5b9-45ce-8b21-d8ac65334826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.664933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373287-c5b9-45ce-8b21-d8ac65334826 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.665217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373287-c5b9-45ce-8b21-d8ac65334826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.665496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373287-c5b9-45ce-8b21-d8ac65334826 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.666227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373287-c5b9-45ce-8b21-d8ac65334826 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:20.666363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 786/1563] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:20.669638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373287-c5b9-45ce-8b21-d8ac65334826 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.669740 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a79de-c5e8-40ae-b2b6-2ec0673a9d20 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.669819 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373287-c5b9-45ce-8b21-d8ac65334826 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.676900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a79de-c5e8-40ae-b2b6-2ec0673a9d20 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.687494 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 778/1564] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:20.687675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0afd4-2d04-4c00-9ad9-65ab3da0cc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.687823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373287-c5b9-45ce-8b21-d8ac65334826 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.687925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0afd4-2d04-4c00-9ad9-65ab3da0cc9f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.687925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0afd4-2d04-4c00-9ad9-65ab3da0cc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.687925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0afd4-2d04-4c00-9ad9-65ab3da0cc9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.687925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0afd4-2d04-4c00-9ad9-65ab3da0cc9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:20.687925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0afd4-2d04-4c00-9ad9-65ab3da0cc9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.687925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0afd4-2d04-4c00-9ad9-65ab3da0cc9f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.694004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373287-c5b9-45ce-8b21-d8ac65334826 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.697491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 787/1565] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:20.698084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0afd4-2d04-4c00-9ad9-65ab3da0cc9f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.706455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a2e09-0668-49e3-b6ae-c066de62b36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.707128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a2e09-0668-49e3-b6ae-c066de62b36b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.707381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0afd4-2d04-4c00-9ad9-65ab3da0cc9f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.707502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a2e09-0668-49e3-b6ae-c066de62b36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.707670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a2e09-0668-49e3-b6ae-c066de62b36b admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.708367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a2e09-0668-49e3-b6ae-c066de62b36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.709452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-958a2e09-0668-49e3-b6ae-c066de62b36b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:35:20.709452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 24 10:35:20.709452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: Jun 24 10:35:20.710672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a2e09-0668-49e3-b6ae-c066de62b36b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.712081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027be162-8b5e-47e7-b21d-2a8968639b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.712708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027be162-8b5e-47e7-b21d-2a8968639b98 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.713017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027be162-8b5e-47e7-b21d-2a8968639b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.713289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027be162-8b5e-47e7-b21d-2a8968639b98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.714302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027be162-8b5e-47e7-b21d-2a8968639b98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.716191 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027be162-8b5e-47e7-b21d-2a8968639b98 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.743790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 779/1566] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:20.743790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a2e09-0668-49e3-b6ae-c066de62b36b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.744106 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 788/1567] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:20.744209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027be162-8b5e-47e7-b21d-2a8968639b98 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.749877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:20.785319 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:20.806289 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a2e09-0668-49e3-b6ae-c066de62b36b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.808065 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027be162-8b5e-47e7-b21d-2a8968639b98 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.810896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb9ab-4128-4e86-8692-e9f5048476e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.811556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb9ab-4128-4e86-8692-e9f5048476e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.811841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb9ab-4128-4e86-8692-e9f5048476e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.812118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb9ab-4128-4e86-8692-e9f5048476e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.812532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11fd92-9a37-4008-93c3-694e57e38690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.812838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44eb9ab-4128-4e86-8692-e9f5048476e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.813237 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11fd92-9a37-4008-93c3-694e57e38690 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.813517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11fd92-9a37-4008-93c3-694e57e38690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.813793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11fd92-9a37-4008-93c3-694e57e38690 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.814565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11fd92-9a37-4008-93c3-694e57e38690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:20.814887 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11fd92-9a37-4008-93c3-694e57e38690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.815217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44eb9ab-4128-4e86-8692-e9f5048476e0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.816540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11fd92-9a37-4008-93c3-694e57e38690 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.829441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 789/1568] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:20.832236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11fd92-9a37-4008-93c3-694e57e38690 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.842251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11fd92-9a37-4008-93c3-694e57e38690 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.844016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 780/1569] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:20.844900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb9ab-4128-4e86-8692-e9f5048476e0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.898406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749c1e0-8bca-46fd-8580-b36a42c57a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.899094 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749c1e0-8bca-46fd-8580-b36a42c57a43 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.899371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749c1e0-8bca-46fd-8580-b36a42c57a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.899626 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749c1e0-8bca-46fd-8580-b36a42c57a43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.900367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749c1e0-8bca-46fd-8580-b36a42c57a43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:20.900669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749c1e0-8bca-46fd-8580-b36a42c57a43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.902351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749c1e0-8bca-46fd-8580-b36a42c57a43 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.910003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb9ab-4128-4e86-8692-e9f5048476e0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.915717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 790/1570] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:20.915717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749c1e0-8bca-46fd-8580-b36a42c57a43 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.919085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cc1f1-076f-4f01-aaa8-65133833763e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.920138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cc1f1-076f-4f01-aaa8-65133833763e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.920434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cc1f1-076f-4f01-aaa8-65133833763e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.920717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cc1f1-076f-4f01-aaa8-65133833763e admin admin] PATH_INFO: `/v3/projects/e41afc594f464d74bd4881e07dd33e45/users/733b7d5058da45089a2133e2fe4695e7/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.925722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749c1e0-8bca-46fd-8580-b36a42c57a43 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:20.930273 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f1407-0dc8-4e1b-bdb9-d8fbeb05818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:20.930919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f1407-0dc8-4e1b-bdb9-d8fbeb05818c admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:20.931165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f1407-0dc8-4e1b-bdb9-d8fbeb05818c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:20.931637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f1407-0dc8-4e1b-bdb9-d8fbeb05818c admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:20.932300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f1407-0dc8-4e1b-bdb9-d8fbeb05818c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.934237 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f1407-0dc8-4e1b-bdb9-d8fbeb05818c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.943002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cc1f1-076f-4f01-aaa8-65133833763e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41afc594f464d74bd4881e07dd33e45, user_id=733b7d5058da45089a2133e2fe4695e7, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:20.944093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-fc0cc1f1-076f-4f01-aaa8-65133833763e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:35:20.944093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 24 10:35:20.944093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: Jun 24 10:35:20.946086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cc1f1-076f-4f01-aaa8-65133833763e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:20.962862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 781/1571] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:20 2022] PUT /identity/v3/projects/e41afc594f464d74bd4881e07dd33e45/users/733b7d5058da45089a2133e2fe4695e7/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:20.963800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cc1f1-076f-4f01-aaa8-65133833763e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:20.969910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 791/1572] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:20.969910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f1407-0dc8-4e1b-bdb9-d8fbeb05818c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.030063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cc1f1-076f-4f01-aaa8-65133833763e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.032586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f1407-0dc8-4e1b-bdb9-d8fbeb05818c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.035020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dea25e-ecd2-4755-8691-b1c48064d836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.035853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dea25e-ecd2-4755-8691-b1c48064d836 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.036083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dea25e-ecd2-4755-8691-b1c48064d836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.036378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dea25e-ecd2-4755-8691-b1c48064d836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.037101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dea25e-ecd2-4755-8691-b1c48064d836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.037617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de68d5b-9213-4f67-85bb-0245a0518aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.038285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de68d5b-9213-4f67-85bb-0245a0518aee admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.038571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de68d5b-9213-4f67-85bb-0245a0518aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.038844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de68d5b-9213-4f67-85bb-0245a0518aee admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.038991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dea25e-ecd2-4755-8691-b1c48064d836 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.039549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de68d5b-9213-4f67-85bb-0245a0518aee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.041531 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de68d5b-9213-4f67-85bb-0245a0518aee admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.057820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:21.066923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 792/1573] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.066923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de68d5b-9213-4f67-85bb-0245a0518aee admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.072308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 782/1574] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:20 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.073129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dea25e-ecd2-4755-8691-b1c48064d836 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.076052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de68d5b-9213-4f67-85bb-0245a0518aee admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.111640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:21.131121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03a429-adaa-4125-a9a5-b2d08a2b9dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.131823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03a429-adaa-4125-a9a5-b2d08a2b9dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.132104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03a429-adaa-4125-a9a5-b2d08a2b9dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.132377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03a429-adaa-4125-a9a5-b2d08a2b9dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.133122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03a429-adaa-4125-a9a5-b2d08a2b9dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.135093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03a429-adaa-4125-a9a5-b2d08a2b9dd8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.140659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dea25e-ecd2-4755-8691-b1c48064d836 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.146871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3f822-50c1-449d-aa52-a6b156d4001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.147581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3f822-50c1-449d-aa52-a6b156d4001c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.147889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3f822-50c1-449d-aa52-a6b156d4001c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.148161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3f822-50c1-449d-aa52-a6b156d4001c admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.148937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3f822-50c1-449d-aa52-a6b156d4001c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.150841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3f822-50c1-449d-aa52-a6b156d4001c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.155358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 793/1575] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:21 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.156129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03a429-adaa-4125-a9a5-b2d08a2b9dd8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.165369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03a429-adaa-4125-a9a5-b2d08a2b9dd8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.171527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee00531-286b-4966-a91f-4a708991c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.172202 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee00531-286b-4966-a91f-4a708991c244 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.172467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee00531-286b-4966-a91f-4a708991c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.172766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee00531-286b-4966-a91f-4a708991c244 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.173452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee00531-286b-4966-a91f-4a708991c244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.175673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee00531-286b-4966-a91f-4a708991c244 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.183297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 783/1576] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:21 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.183297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3f822-50c1-449d-aa52-a6b156d4001c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.201355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 794/1577] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:21 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.201355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe65de-41ef-49e0-a12e-c89bdc5366df None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.201355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe65de-41ef-49e0-a12e-c89bdc5366df None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.201355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe65de-41ef-49e0-a12e-c89bdc5366df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.219920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-41fe65de-41ef-49e0-a12e-c89bdc5366df None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:21.221220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-41fe65de-41ef-49e0-a12e-c89bdc5366df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:21.244316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3f822-50c1-449d-aa52-a6b156d4001c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.248983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e191615-9792-4b03-a745-511ab5aa1fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.249652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e191615-9792-4b03-a745-511ab5aa1fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.249939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e191615-9792-4b03-a745-511ab5aa1fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.250212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e191615-9792-4b03-a745-511ab5aa1fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.250898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e191615-9792-4b03-a745-511ab5aa1fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.253283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e191615-9792-4b03-a745-511ab5aa1fc3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.284535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 784/1578] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:21 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.284535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e191615-9792-4b03-a745-511ab5aa1fc3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.289761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 795/1579] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:21.290974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe65de-41ef-49e0-a12e-c89bdc5366df None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.300799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe65de-41ef-49e0-a12e-c89bdc5366df None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.350848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e191615-9792-4b03-a745-511ab5aa1fc3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.355622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ee51-0a2b-48b3-af4a-d9c4591de061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.356338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ee51-0a2b-48b3-af4a-d9c4591de061 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.356614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ee51-0a2b-48b3-af4a-d9c4591de061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.356904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ee51-0a2b-48b3-af4a-d9c4591de061 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.357652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550ee51-0a2b-48b3-af4a-d9c4591de061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.357898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2f062-54f9-4511-8ae3-719f9dd373f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.358694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f062-54f9-4511-8ae3-719f9dd373f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.358985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f062-54f9-4511-8ae3-719f9dd373f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.359317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f062-54f9-4511-8ae3-719f9dd373f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.359577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550ee51-0a2b-48b3-af4a-d9c4591de061 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.360044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2f062-54f9-4511-8ae3-719f9dd373f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.362387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2f062-54f9-4511-8ae3-719f9dd373f9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.376266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 796/1580] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.376266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2f062-54f9-4511-8ae3-719f9dd373f9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.380583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 785/1581] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:21 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.381767 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ee51-0a2b-48b3-af4a-d9c4591de061 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.385582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2f062-54f9-4511-8ae3-719f9dd373f9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.390964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ee51-0a2b-48b3-af4a-d9c4591de061 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.391757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f780-9425-40e6-b869-8c6028056331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.392504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f780-9425-40e6-b869-8c6028056331 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.392936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f780-9425-40e6-b869-8c6028056331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.393671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f780-9425-40e6-b869-8c6028056331 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.394355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f780-9425-40e6-b869-8c6028056331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.395501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb8762-e0af-421b-a9f5-2ede4209da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.396140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb8762-e0af-421b-a9f5-2ede4209da1d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.396288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f780-9425-40e6-b869-8c6028056331 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.396389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb8762-e0af-421b-a9f5-2ede4209da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.396689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb8762-e0af-421b-a9f5-2ede4209da1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.397448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb8762-e0af-421b-a9f5-2ede4209da1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.399014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb8762-e0af-421b-a9f5-2ede4209da1d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.404026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:21.411645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 786/1582] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.415585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb8762-e0af-421b-a9f5-2ede4209da1d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.420928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 797/1583] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:21 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.422586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f780-9425-40e6-b869-8c6028056331 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.426467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb8762-e0af-421b-a9f5-2ede4209da1d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.431286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc459914-d1ad-4214-b386-68cc386d152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.431978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459914-d1ad-4214-b386-68cc386d152c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.432351 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459914-d1ad-4214-b386-68cc386d152c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.432481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f780-9425-40e6-b869-8c6028056331 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.432661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459914-d1ad-4214-b386-68cc386d152c admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.433394 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc459914-d1ad-4214-b386-68cc386d152c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.435555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc459914-d1ad-4214-b386-68cc386d152c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.436975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a37514-3bec-4791-9368-a053498e904a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.437633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a37514-3bec-4791-9368-a053498e904a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.437920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a37514-3bec-4791-9368-a053498e904a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.438198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a37514-3bec-4791-9368-a053498e904a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.438930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a37514-3bec-4791-9368-a053498e904a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.440605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a37514-3bec-4791-9368-a053498e904a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.454561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 798/1584] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.456194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a37514-3bec-4791-9368-a053498e904a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.460911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 787/1585] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:21 2022] POST /identity/v3/users => generated 291 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.461868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc459914-d1ad-4214-b386-68cc386d152c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.467849 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:21.469025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a37514-3bec-4791-9368-a053498e904a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.474885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5583e0-3f21-4bc0-982c-e896ccffc5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.475556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5583e0-3f21-4bc0-982c-e896ccffc5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.475840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5583e0-3f21-4bc0-982c-e896ccffc5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.476160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5583e0-3f21-4bc0-982c-e896ccffc5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.476835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5583e0-3f21-4bc0-982c-e896ccffc5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.478774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5583e0-3f21-4bc0-982c-e896ccffc5d5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.500578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 799/1586] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:21 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:21.501463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5583e0-3f21-4bc0-982c-e896ccffc5d5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.512136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5583e0-3f21-4bc0-982c-e896ccffc5d5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.516831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d21a0-36af-43ae-913e-d746850be1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.517713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d21a0-36af-43ae-913e-d746850be1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.517999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d21a0-36af-43ae-913e-d746850be1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.518278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d21a0-36af-43ae-913e-d746850be1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.519022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d21a0-36af-43ae-913e-d746850be1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.520648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d21a0-36af-43ae-913e-d746850be1e1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.522503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc459914-d1ad-4214-b386-68cc386d152c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.526109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e2e00-a347-4010-8a1d-d3b8ae26d143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.526773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e2e00-a347-4010-8a1d-d3b8ae26d143 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.527063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e2e00-a347-4010-8a1d-d3b8ae26d143 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.527344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e2e00-a347-4010-8a1d-d3b8ae26d143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.531003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 800/1587] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.531872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d21a0-36af-43ae-913e-d746850be1e1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.537467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e2e00-a347-4010-8a1d-d3b8ae26d143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.538930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e2e00-a347-4010-8a1d-d3b8ae26d143 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.541869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d21a0-36af-43ae-913e-d746850be1e1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.546316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bd83e-433b-41d6-ac34-044e65a80cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.547019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bd83e-433b-41d6-ac34-044e65a80cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.547247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bd83e-433b-41d6-ac34-044e65a80cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.547527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bd83e-433b-41d6-ac34-044e65a80cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.548242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bd83e-433b-41d6-ac34-044e65a80cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.548950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 788/1588] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:21.550015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e2e00-a347-4010-8a1d-d3b8ae26d143 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.550493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bd83e-433b-41d6-ac34-044e65a80cbd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.560474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e2e00-a347-4010-8a1d-d3b8ae26d143 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.564785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 801/1589] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.564785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bd83e-433b-41d6-ac34-044e65a80cbd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.566503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc76bfc-f6eb-4949-a239-63d0f7802bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.567318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc76bfc-f6eb-4949-a239-63d0f7802bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.567600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc76bfc-f6eb-4949-a239-63d0f7802bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.567882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc76bfc-f6eb-4949-a239-63d0f7802bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.568611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc76bfc-f6eb-4949-a239-63d0f7802bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.570257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc76bfc-f6eb-4949-a239-63d0f7802bb5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.575161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bd83e-433b-41d6-ac34-044e65a80cbd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.581068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62693efc-e808-4e9b-bf84-758111a93e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.581644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 789/1590] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.581770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62693efc-e808-4e9b-bf84-758111a93e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.582146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62693efc-e808-4e9b-bf84-758111a93e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.582475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc76bfc-f6eb-4949-a239-63d0f7802bb5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.582585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62693efc-e808-4e9b-bf84-758111a93e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.583293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62693efc-e808-4e9b-bf84-758111a93e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.585297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62693efc-e808-4e9b-bf84-758111a93e09 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.594977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc76bfc-f6eb-4949-a239-63d0f7802bb5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.599673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 802/1591] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.599673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62693efc-e808-4e9b-bf84-758111a93e09 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.603497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660f02c-1c73-486e-8183-2459583e59f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.604430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660f02c-1c73-486e-8183-2459583e59f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.604430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660f02c-1c73-486e-8183-2459583e59f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.604709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660f02c-1c73-486e-8183-2459583e59f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.605485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660f02c-1c73-486e-8183-2459583e59f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.607406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660f02c-1c73-486e-8183-2459583e59f6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.608516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62693efc-e808-4e9b-bf84-758111a93e09 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.614239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9eac7-337b-4fdc-81bd-c5bcdde7c86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.614810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9eac7-337b-4fdc-81bd-c5bcdde7c86a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.615085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9eac7-337b-4fdc-81bd-c5bcdde7c86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.615372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9eac7-337b-4fdc-81bd-c5bcdde7c86a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.616219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9eac7-337b-4fdc-81bd-c5bcdde7c86a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.617862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9eac7-337b-4fdc-81bd-c5bcdde7c86a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.619224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 790/1592] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.620069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660f02c-1c73-486e-8183-2459583e59f6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.631839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660f02c-1c73-486e-8183-2459583e59f6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.637321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 803/1593] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.637321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9eac7-337b-4fdc-81bd-c5bcdde7c86a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.637948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7269f3-93ba-4a51-b4e3-a9a647d3bd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.638631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7269f3-93ba-4a51-b4e3-a9a647d3bd1a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.638866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7269f3-93ba-4a51-b4e3-a9a647d3bd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.639153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7269f3-93ba-4a51-b4e3-a9a647d3bd1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.639894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7269f3-93ba-4a51-b4e3-a9a647d3bd1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.641781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7269f3-93ba-4a51-b4e3-a9a647d3bd1a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.647621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9eac7-337b-4fdc-81bd-c5bcdde7c86a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.653615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64439a-6729-436e-9316-bc942947f9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.654435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 791/1594] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.654768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64439a-6729-436e-9316-bc942947f9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.655110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64439a-6729-436e-9316-bc942947f9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.655561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64439a-6729-436e-9316-bc942947f9e6 admin admin] PATH_INFO: `/v3/projects/98f3e7ee3a4648fcb630682d1ab59dba/users/b1a6ca70bb944325a9e9424853a0bc48/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.656423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7269f3-93ba-4a51-b4e3-a9a647d3bd1a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.666819 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7269f3-93ba-4a51-b4e3-a9a647d3bd1a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.671527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30756e1-a009-4b67-84ba-d9d6a776dbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.672168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30756e1-a009-4b67-84ba-d9d6a776dbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.672511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30756e1-a009-4b67-84ba-d9d6a776dbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.672775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30756e1-a009-4b67-84ba-d9d6a776dbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.673348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64439a-6729-436e-9316-bc942947f9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f3e7ee3a4648fcb630682d1ab59dba, user_id=b1a6ca70bb944325a9e9424853a0bc48, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.673469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30756e1-a009-4b67-84ba-d9d6a776dbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.675102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30756e1-a009-4b67-84ba-d9d6a776dbb9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.676870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64439a-6729-436e-9316-bc942947f9e6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.686384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 792/1595] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.690623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30756e1-a009-4b67-84ba-d9d6a776dbb9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.692841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 804/1596] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:21 2022] PUT /identity/v3/projects/98f3e7ee3a4648fcb630682d1ab59dba/users/b1a6ca70bb944325a9e9424853a0bc48/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:21.694126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64439a-6729-436e-9316-bc942947f9e6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.706911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30756e1-a009-4b67-84ba-d9d6a776dbb9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.772087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64439a-6729-436e-9316-bc942947f9e6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.777532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c57670-a775-4800-8dd9-bc817274da60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.778212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c57670-a775-4800-8dd9-bc817274da60 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.778494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c57670-a775-4800-8dd9-bc817274da60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.778766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c57670-a775-4800-8dd9-bc817274da60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.779624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c57670-a775-4800-8dd9-bc817274da60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.781425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c57670-a775-4800-8dd9-bc817274da60 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.781751 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58546386-8145-4e63-9035-2bb284115760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.782758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58546386-8145-4e63-9035-2bb284115760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.783208 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58546386-8145-4e63-9035-2bb284115760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.783892 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58546386-8145-4e63-9035-2bb284115760 admin admin] PATH_INFO: `/v3/projects/281651371cba4d4a9454b0747947cc83/users/3e4ed08c1c81453abaa6c1dd826f0258/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.795975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 805/1597] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.796758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c57670-a775-4800-8dd9-bc817274da60 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.804753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58546386-8145-4e63-9035-2bb284115760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281651371cba4d4a9454b0747947cc83, user_id=3e4ed08c1c81453abaa6c1dd826f0258, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.807347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58546386-8145-4e63-9035-2bb284115760 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.810239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c57670-a775-4800-8dd9-bc817274da60 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.816846 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa539e-aed0-44ec-b43b-62974a430c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.817892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa539e-aed0-44ec-b43b-62974a430c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.818190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa539e-aed0-44ec-b43b-62974a430c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.818627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa539e-aed0-44ec-b43b-62974a430c6a admin admin] PATH_INFO: `/v3/projects/8539804dc27f42fe88833acb7b6cfa87/users/9bb081bd64964cabb5986eb7cb976617/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.824103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 793/1598] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:21 2022] PUT /identity/v3/projects/281651371cba4d4a9454b0747947cc83/users/3e4ed08c1c81453abaa6c1dd826f0258/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:21.824878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58546386-8145-4e63-9035-2bb284115760 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.841699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa539e-aed0-44ec-b43b-62974a430c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8539804dc27f42fe88833acb7b6cfa87, user_id=9bb081bd64964cabb5986eb7cb976617, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.844387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa539e-aed0-44ec-b43b-62974a430c6a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.863933 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 806/1599] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:21 2022] PUT /identity/v3/projects/8539804dc27f42fe88833acb7b6cfa87/users/9bb081bd64964cabb5986eb7cb976617/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:21.864765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa539e-aed0-44ec-b43b-62974a430c6a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.883901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:21.899160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:21.902614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58546386-8145-4e63-9035-2bb284115760 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.941137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa539e-aed0-44ec-b43b-62974a430c6a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.946462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8b988-b669-4d7a-b989-f739a16d9ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.947141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b988-b669-4d7a-b989-f739a16d9ada admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.947425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b988-b669-4d7a-b989-f739a16d9ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.947703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b988-b669-4d7a-b989-f739a16d9ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.948463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8b988-b669-4d7a-b989-f739a16d9ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.950386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8b988-b669-4d7a-b989-f739a16d9ada admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:21.967998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 807/1600] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:21.967998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8b988-b669-4d7a-b989-f739a16d9ada admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:21.978948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8b988-b669-4d7a-b989-f739a16d9ada admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:21.983836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9d4ca-55cd-43fc-bc15-feb587b9cbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.984758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9d4ca-55cd-43fc-bc15-feb587b9cbbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.985160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9d4ca-55cd-43fc-bc15-feb587b9cbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.985751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9d4ca-55cd-43fc-bc15-feb587b9cbbc admin admin] PATH_INFO: `/v3/projects/2c0d7938e98147cc8dd73e1a89fd2c31/users/4c652ef9b3ea4f29aa68c0e8ed4c16f3/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.993802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c24ec1-28b9-4d4d-8f81-6ffc7922bd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:21.994937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c24ec1-28b9-4d4d-8f81-6ffc7922bd79 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:21.995225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c24ec1-28b9-4d4d-8f81-6ffc7922bd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:21.995503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c24ec1-28b9-4d4d-8f81-6ffc7922bd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:21.996249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c24ec1-28b9-4d4d-8f81-6ffc7922bd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:21.997942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c24ec1-28b9-4d4d-8f81-6ffc7922bd79 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:22.001307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9d4ca-55cd-43fc-bc15-feb587b9cbbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0d7938e98147cc8dd73e1a89fd2c31, user_id=4c652ef9b3ea4f29aa68c0e8ed4c16f3, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:22.004207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9d4ca-55cd-43fc-bc15-feb587b9cbbc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:22.011523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 794/1601] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:22.011523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57475f-1a86-4ab2-a665-bdb2f8f3b83b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.011523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57475f-1a86-4ab2-a665-bdb2f8f3b83b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.011523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57475f-1a86-4ab2-a665-bdb2f8f3b83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.026525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 808/1602] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:21 2022] PUT /identity/v3/projects/2c0d7938e98147cc8dd73e1a89fd2c31/users/4c652ef9b3ea4f29aa68c0e8ed4c16f3/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:22.027691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9d4ca-55cd-43fc-bc15-feb587b9cbbc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:22.041368 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ca57475f-1a86-4ab2-a665-bdb2f8f3b83b None None] MFA Rules not processed for user `b1a6ca70bb944325a9e9424853a0bc48`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:22.043023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ca57475f-1a86-4ab2-a665-bdb2f8f3b83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.102281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9d4ca-55cd-43fc-bc15-feb587b9cbbc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:22.107934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7c51e-0981-425c-8d30-090ee4db8252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:22.108867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7c51e-0981-425c-8d30-090ee4db8252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.109149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7c51e-0981-425c-8d30-090ee4db8252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.109420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7c51e-0981-425c-8d30-090ee4db8252 admin admin] PATH_INFO: `/v3/projects/dee3d0adc7084828b9ed954edee54f6a/users/16daeab4accb4eeca4100b55dc9bcaa2/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.120970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 795/1603] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:22.125473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d3ea0-f9a9-474c-bbf4-dce51bb1a044 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.125473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d3ea0-f9a9-474c-bbf4-dce51bb1a044 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.125473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d3ea0-f9a9-474c-bbf4-dce51bb1a044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.130648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7c51e-0981-425c-8d30-090ee4db8252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee3d0adc7084828b9ed954edee54f6a, user_id=16daeab4accb4eeca4100b55dc9bcaa2, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:22.133670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7c51e-0981-425c-8d30-090ee4db8252 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:22.149843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-568d3ea0-f9a9-474c-bbf4-dce51bb1a044 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:22.151172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-568d3ea0-f9a9-474c-bbf4-dce51bb1a044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.156805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 809/1604] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:22 2022] PUT /identity/v3/projects/dee3d0adc7084828b9ed954edee54f6a/users/16daeab4accb4eeca4100b55dc9bcaa2/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:22.159719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4e320-65cd-4c00-954c-5ada885a0de3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.160618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4e320-65cd-4c00-954c-5ada885a0de3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.161368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4e320-65cd-4c00-954c-5ada885a0de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.195120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e2f4e320-65cd-4c00-954c-5ada885a0de3 None None] MFA Rules not processed for user `9bb081bd64964cabb5986eb7cb976617`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:22.196361 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e2f4e320-65cd-4c00-954c-5ada885a0de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.219873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 796/1605] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:22.219873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d3ea0-f9a9-474c-bbf4-dce51bb1a044 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:22.230498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d3ea0-f9a9-474c-bbf4-dce51bb1a044 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:22.242467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1facc498-868f-4655-94eb-119bc668a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:22.244035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facc498-868f-4655-94eb-119bc668a011 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.244807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facc498-868f-4655-94eb-119bc668a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.244807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facc498-868f-4655-94eb-119bc668a011 admin admin] PATH_INFO: `/v3/projects/d8cc166ec03349b3b2b0d69840886bd9/users/960b18fe180f4185b291da1d9c18a2fa/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.264979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1facc498-868f-4655-94eb-119bc668a011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cc166ec03349b3b2b0d69840886bd9, user_id=960b18fe180f4185b291da1d9c18a2fa, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:22.268297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 810/1606] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:22.269689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1facc498-868f-4655-94eb-119bc668a011 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:22.269943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4e320-65cd-4c00-954c-5ada885a0de3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:22.275474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:22.282000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4e320-65cd-4c00-954c-5ada885a0de3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:22.303112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf4231-2501-483e-ab00-1720212a0d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:22.304132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf4231-2501-483e-ab00-1720212a0d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.305253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf4231-2501-483e-ab00-1720212a0d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.305756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf4231-2501-483e-ab00-1720212a0d91 admin admin] PATH_INFO: `/v3/projects/dcbf3094963e41b9bb17026f4eeca3ca/users/d5354b3fb3664800b2f247a34d3de386/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.311882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 797/1607] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:22 2022] PUT /identity/v3/projects/d8cc166ec03349b3b2b0d69840886bd9/users/960b18fe180f4185b291da1d9c18a2fa/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:22.316207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93bf3-dee2-45db-ab10-cb433ac09344 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.316507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93bf3-dee2-45db-ab10-cb433ac09344 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.317032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:22.319094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93bf3-dee2-45db-ab10-cb433ac09344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.328291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf4231-2501-483e-ab00-1720212a0d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbf3094963e41b9bb17026f4eeca3ca, user_id=d5354b3fb3664800b2f247a34d3de386, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:22.331670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf4231-2501-483e-ab00-1720212a0d91 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:22.355055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 811/1608] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:22 2022] PUT /identity/v3/projects/dcbf3094963e41b9bb17026f4eeca3ca/users/d5354b3fb3664800b2f247a34d3de386/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:22.358070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a5faa-0183-408a-8bb9-436f46362069 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.358902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a5faa-0183-408a-8bb9-436f46362069 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.359568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a5faa-0183-408a-8bb9-436f46362069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.372638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d8a93bf3-dee2-45db-ab10-cb433ac09344 None None] MFA Rules not processed for user `4c652ef9b3ea4f29aa68c0e8ed4c16f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:22.374234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d8a93bf3-dee2-45db-ab10-cb433ac09344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.384089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1b8a5faa-0183-408a-8bb9-436f46362069 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:22.385965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1b8a5faa-0183-408a-8bb9-436f46362069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.465257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 812/1609] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:22.467282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8487-7d00-4873-942e-eabfdcdd894a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.467852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8487-7d00-4873-942e-eabfdcdd894a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.468334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8487-7d00-4873-942e-eabfdcdd894a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.470364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 813/1610] 10.222.0.69 () {54 vars in 961 bytes} [Fri Jun 24 10:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:35:22.471365 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b8487-7d00-4873-942e-eabfdcdd894a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:22.486831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 798/1611] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:22.492299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ba657-e38e-4e62-a3d3-75d81088dda5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.492595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ba657-e38e-4e62-a3d3-75d81088dda5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.492925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ba657-e38e-4e62-a3d3-75d81088dda5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.499821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 799/1612] 10.222.0.69 () {54 vars in 961 bytes} [Fri Jun 24 10:35:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:35:22.501685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caef92-3135-4c90-baca-25f539e114da None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.502038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caef92-3135-4c90-baca-25f539e114da None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.503666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caef92-3135-4c90-baca-25f539e114da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.530205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-73caef92-3135-4c90-baca-25f539e114da None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:22.531803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-73caef92-3135-4c90-baca-25f539e114da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.542138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b8487-7d00-4873-942e-eabfdcdd894a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:22.542828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 800/1613] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:22.544464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527c7d4-7417-416d-ac9f-27f6b5a8e7ec None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.545172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527c7d4-7417-416d-ac9f-27f6b5a8e7ec None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.545812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527c7d4-7417-416d-ac9f-27f6b5a8e7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.546563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0e216-7a37-46d2-bd9e-bd8d00c48e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:22.547616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0e216-7a37-46d2-bd9e-bd8d00c48e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.548015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0e216-7a37-46d2-bd9e-bd8d00c48e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.548393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0e216-7a37-46d2-bd9e-bd8d00c48e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.560304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0e216-7a37-46d2-bd9e-bd8d00c48e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:22.562298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0e216-7a37-46d2-bd9e-bd8d00c48e71 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:22.575488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 814/1614] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:22.576408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0e216-7a37-46d2-bd9e-bd8d00c48e71 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:22.578576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bfb0e216-7a37-46d2-bd9e-bd8d00c48e71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.588040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9527c7d4-7417-416d-ac9f-27f6b5a8e7ec None None] MFA Rules not processed for user `d5354b3fb3664800b2f247a34d3de386`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:22.590042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9527c7d4-7417-416d-ac9f-27f6b5a8e7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.593496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0e216-7a37-46d2-bd9e-bd8d00c48e71 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:22.598697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b65a8-16fc-436f-939e-81579534456c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:22.599517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b65a8-16fc-436f-939e-81579534456c service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.599926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b65a8-16fc-436f-939e-81579534456c service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.600305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b65a8-16fc-436f-939e-81579534456c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.610722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b65a8-16fc-436f-939e-81579534456c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:22.614296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b65a8-16fc-436f-939e-81579534456c service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:22.626756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 815/1615] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:22.629645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e5265-e36f-404d-b5c8-398dae03b20b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.630016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e5265-e36f-404d-b5c8-398dae03b20b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.630571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e5265-e36f-404d-b5c8-398dae03b20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.652972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9d8e5265-e36f-404d-b5c8-398dae03b20b None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:22.654665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9d8e5265-e36f-404d-b5c8-398dae03b20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.699367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 801/1616] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:22.699367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527c7d4-7417-416d-ac9f-27f6b5a8e7ec None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:22.713098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527c7d4-7417-416d-ac9f-27f6b5a8e7ec None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:22.719620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e26256-b3e5-4a92-9a5c-41b260ee4535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:22.720450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e26256-b3e5-4a92-9a5c-41b260ee4535 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.721231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e26256-b3e5-4a92-9a5c-41b260ee4535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.721517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e26256-b3e5-4a92-9a5c-41b260ee4535 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.722230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e26256-b3e5-4a92-9a5c-41b260ee4535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:22.724802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e26256-b3e5-4a92-9a5c-41b260ee4535 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:22.771511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 802/1617] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/projects => generated 405 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:22.773501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18142ecd-9e34-4404-a05f-92b240f3c2e7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.773800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18142ecd-9e34-4404-a05f-92b240f3c2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.774279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18142ecd-9e34-4404-a05f-92b240f3c2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:22.797243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-18142ecd-9e34-4404-a05f-92b240f3c2e7 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:22.799482 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 816/1618] 10.222.0.69 () {58 vars in 1058 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:22.799626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-18142ecd-9e34-4404-a05f-92b240f3c2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:22.811114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e5265-e36f-404d-b5c8-398dae03b20b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:22.834483 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e5265-e36f-404d-b5c8-398dae03b20b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:22.849692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:22.862349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:22.944272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 803/1619] 10.222.0.69 () {58 vars in 1058 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:22.945340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18142ecd-9e34-4404-a05f-92b240f3c2e7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:22.969392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ea7a8-0ab4-428a-9005-1dffb5e3655c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:22.970203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ea7a8-0ab4-428a-9005-1dffb5e3655c service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:22.970734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ea7a8-0ab4-428a-9005-1dffb5e3655c service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:22.971017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ea7a8-0ab4-428a-9005-1dffb5e3655c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.074668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18142ecd-9e34-4404-a05f-92b240f3c2e7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:23.088078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c5fdc-41b7-4b18-8ab4-49ef85750817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:23.088903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c5fdc-41b7-4b18-8ab4-49ef85750817 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:23.089599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c5fdc-41b7-4b18-8ab4-49ef85750817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:23.090045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c5fdc-41b7-4b18-8ab4-49ef85750817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.090768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c5fdc-41b7-4b18-8ab4-49ef85750817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.095858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c5fdc-41b7-4b18-8ab4-49ef85750817 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.098654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ea7a8-0ab4-428a-9005-1dffb5e3655c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.106280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ea7a8-0ab4-428a-9005-1dffb5e3655c service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.128844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 817/1620] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:23.134123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ea7a8-0ab4-428a-9005-1dffb5e3655c service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:23.159354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ea7a8-0ab4-428a-9005-1dffb5e3655c service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:23.161866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 804/1621] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:22 2022] POST /identity/v3/projects => generated 381 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:23.162585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c5fdc-41b7-4b18-8ab4-49ef85750817 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:23.281734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67285f10-9b5d-4158-8049-dbaa8cffbafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:23.282544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67285f10-9b5d-4158-8049-dbaa8cffbafe admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:23.282831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67285f10-9b5d-4158-8049-dbaa8cffbafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:23.283201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67285f10-9b5d-4158-8049-dbaa8cffbafe admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.284207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67285f10-9b5d-4158-8049-dbaa8cffbafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.286660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67285f10-9b5d-4158-8049-dbaa8cffbafe admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.318287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 818/1622] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:23 2022] POST /identity/v3/users => generated 309 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:23.322501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67285f10-9b5d-4158-8049-dbaa8cffbafe admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:23.334661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c5fdc-41b7-4b18-8ab4-49ef85750817 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:23.335974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67285f10-9b5d-4158-8049-dbaa8cffbafe admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:23.343203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad5a4d-00fc-43ae-8a66-23beffb88725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:23.344109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad5a4d-00fc-43ae-8a66-23beffb88725 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:23.344401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad5a4d-00fc-43ae-8a66-23beffb88725 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:23.344681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad5a4d-00fc-43ae-8a66-23beffb88725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.350107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01096a-a2fd-4b21-97f5-3accc08ea624 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:23.350882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01096a-a2fd-4b21-97f5-3accc08ea624 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:23.351202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01096a-a2fd-4b21-97f5-3accc08ea624 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:23.351488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01096a-a2fd-4b21-97f5-3accc08ea624 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.468145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:23.494406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad5a4d-00fc-43ae-8a66-23beffb88725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.496305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad5a4d-00fc-43ae-8a66-23beffb88725 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.501962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01096a-a2fd-4b21-97f5-3accc08ea624 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.503584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01096a-a2fd-4b21-97f5-3accc08ea624 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.510965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:23.516477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 805/1623] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:23.519299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01096a-a2fd-4b21-97f5-3accc08ea624 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:23.537494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 819/1624] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:23.539776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01096a-a2fd-4b21-97f5-3accc08ea624 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:23.539930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad5a4d-00fc-43ae-8a66-23beffb88725 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:23.548161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8955023-ec03-45da-9197-c3d66e21be9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:23.549355 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955023-ec03-45da-9197-c3d66e21be9a admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:23.549843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955023-ec03-45da-9197-c3d66e21be9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:23.549997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955023-ec03-45da-9197-c3d66e21be9a admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.551197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8955023-ec03-45da-9197-c3d66e21be9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.560191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8955023-ec03-45da-9197-c3d66e21be9a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.574913 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad5a4d-00fc-43ae-8a66-23beffb88725 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:23.606375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bbef6-849c-41e7-9ace-6a60e8df0020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:23.606946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 806/1625] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:23 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:23.607117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bbef6-849c-41e7-9ace-6a60e8df0020 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:23.607409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bbef6-849c-41e7-9ace-6a60e8df0020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:23.607687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bbef6-849c-41e7-9ace-6a60e8df0020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.608687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bbef6-849c-41e7-9ace-6a60e8df0020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.611798 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8955023-ec03-45da-9197-c3d66e21be9a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:23.622265 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bbef6-849c-41e7-9ace-6a60e8df0020 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.649462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8955023-ec03-45da-9197-c3d66e21be9a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:23.662749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 820/1626] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:23.663220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc3f8f-f591-4596-8fb7-f3b79bf6d0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:23.664242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc3f8f-f591-4596-8fb7-f3b79bf6d0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:23.664627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc3f8f-f591-4596-8fb7-f3b79bf6d0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:23.665024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc3f8f-f591-4596-8fb7-f3b79bf6d0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.665889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc3f8f-f591-4596-8fb7-f3b79bf6d0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.667091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bbef6-849c-41e7-9ace-6a60e8df0020 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:23.668386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc3f8f-f591-4596-8fb7-f3b79bf6d0c2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.695383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 807/1627] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:23.699326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc3f8f-f591-4596-8fb7-f3b79bf6d0c2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:23.705056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bbef6-849c-41e7-9ace-6a60e8df0020 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:23.720445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc3f8f-f591-4596-8fb7-f3b79bf6d0c2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:23.724357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d90d25-3749-46c1-a120-721f33af42ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:23.725343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d90d25-3749-46c1-a120-721f33af42ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:23.725624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d90d25-3749-46c1-a120-721f33af42ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:23.725902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d90d25-3749-46c1-a120-721f33af42ba admin admin] PATH_INFO: `/v3/projects/26584a16924744169ca6ffd7fd2a8cda/users/e004515e2ef847a793cd77fbe1e29d75/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.735041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0aa376-3906-4351-a43a-06bc8680fc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:23.736148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aa376-3906-4351-a43a-06bc8680fc25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:23.736696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aa376-3906-4351-a43a-06bc8680fc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:23.738657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aa376-3906-4351-a43a-06bc8680fc25 admin admin] PATH_INFO: `/v3/projects/7b88cdc82ee04393b8af77d1958590ce/users/29dd9bd37d4b4f46bc5d1043361eb070/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:23.778828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d90d25-3749-46c1-a120-721f33af42ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=26584a16924744169ca6ffd7fd2a8cda, user_id=e004515e2ef847a793cd77fbe1e29d75, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.800214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0aa376-3906-4351-a43a-06bc8680fc25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b88cdc82ee04393b8af77d1958590ce, user_id=29dd9bd37d4b4f46bc5d1043361eb070, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:23.871853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d90d25-3749-46c1-a120-721f33af42ba admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.876441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0aa376-3906-4351-a43a-06bc8680fc25 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:23.876441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 808/1628] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:23 2022] PUT /identity/v3/projects/7b88cdc82ee04393b8af77d1958590ce/users/29dd9bd37d4b4f46bc5d1043361eb070/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:23.876441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0aa376-3906-4351-a43a-06bc8680fc25 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:23.877157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 821/1629] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:23 2022] PUT /identity/v3/projects/26584a16924744169ca6ffd7fd2a8cda/users/e004515e2ef847a793cd77fbe1e29d75/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:23.881363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d90d25-3749-46c1-a120-721f33af42ba admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:24.060522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0aa376-3906-4351-a43a-06bc8680fc25 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:24.089339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d90d25-3749-46c1-a120-721f33af42ba admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:24.104571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ad279-a0a8-46b2-830b-c24ab9d07f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:24.105609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ad279-a0a8-46b2-830b-c24ab9d07f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.106043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ad279-a0a8-46b2-830b-c24ab9d07f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.106469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ad279-a0a8-46b2-830b-c24ab9d07f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.107792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ad279-a0a8-46b2-830b-c24ab9d07f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:24.109722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ad279-a0a8-46b2-830b-c24ab9d07f64 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:24.125398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 822/1630] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:24.131095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bb172-9932-4f18-bf65-3c8bb8addad9 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.132113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bb172-9932-4f18-bf65-3c8bb8addad9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.132113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bb172-9932-4f18-bf65-3c8bb8addad9 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.141920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 823/1631] 10.222.0.69 () {54 vars in 912 bytes} [Fri Jun 24 10:35:24 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:35:24.145904 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bb172-9932-4f18-bf65-3c8bb8addad9 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:24.174712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bb172-9932-4f18-bf65-3c8bb8addad9 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:24.188659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547dc61-55f6-4703-aaff-dbec90cb30f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:24.189426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547dc61-55f6-4703-aaff-dbec90cb30f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.189695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232ecbe-9536-4169-9b59-43d8efb79bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:24.189896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547dc61-55f6-4703-aaff-dbec90cb30f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.190001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547dc61-55f6-4703-aaff-dbec90cb30f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.190694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547dc61-55f6-4703-aaff-dbec90cb30f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:24.191194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232ecbe-9536-4169-9b59-43d8efb79bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.191648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232ecbe-9536-4169-9b59-43d8efb79bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.192089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232ecbe-9536-4169-9b59-43d8efb79bb6 admin admin] PATH_INFO: `/v3/projects/26584a16924744169ca6ffd7fd2a8cda/users/e004515e2ef847a793cd77fbe1e29d75/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.192302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547dc61-55f6-4703-aaff-dbec90cb30f2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:24.194476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232ecbe-9536-4169-9b59-43d8efb79bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26584a16924744169ca6ffd7fd2a8cda, user_id=e004515e2ef847a793cd77fbe1e29d75, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:24.197350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232ecbe-9536-4169-9b59-43d8efb79bb6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:24.218100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 809/1632] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:24.218100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa5e54-4f9d-4229-8a2a-4d7fd369caf4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.218100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa5e54-4f9d-4229-8a2a-4d7fd369caf4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.218100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa5e54-4f9d-4229-8a2a-4d7fd369caf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.224708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 824/1633] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:24 2022] PUT /identity/v3/projects/26584a16924744169ca6ffd7fd2a8cda/users/e004515e2ef847a793cd77fbe1e29d75/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:24.225587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232ecbe-9536-4169-9b59-43d8efb79bb6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:24.266757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2bfa5e54-4f9d-4229-8a2a-4d7fd369caf4 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:24.269840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2bfa5e54-4f9d-4229-8a2a-4d7fd369caf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:24.331019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232ecbe-9536-4169-9b59-43d8efb79bb6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:24.336759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366016e2-6e4d-4e9c-8e97-c75812ac75b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:24.337831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366016e2-6e4d-4e9c-8e97-c75812ac75b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.338117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366016e2-6e4d-4e9c-8e97-c75812ac75b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.338696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366016e2-6e4d-4e9c-8e97-c75812ac75b4 admin admin] PATH_INFO: `/v3/projects/7b88cdc82ee04393b8af77d1958590ce/users/29dd9bd37d4b4f46bc5d1043361eb070/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.342663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366016e2-6e4d-4e9c-8e97-c75812ac75b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b88cdc82ee04393b8af77d1958590ce, user_id=29dd9bd37d4b4f46bc5d1043361eb070, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:24.347425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366016e2-6e4d-4e9c-8e97-c75812ac75b4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:24.363242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 825/1634] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:24 2022] PUT /identity/v3/projects/7b88cdc82ee04393b8af77d1958590ce/users/29dd9bd37d4b4f46bc5d1043361eb070/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:24.365014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c11d15-0d80-4674-a9ac-0449b7606ff2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.365512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c11d15-0d80-4674-a9ac-0449b7606ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.366043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c11d15-0d80-4674-a9ac-0449b7606ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.396610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:24.431447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-30c11d15-0d80-4674-a9ac-0449b7606ff2 None None] MFA Rules not processed for user `e004515e2ef847a793cd77fbe1e29d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:24.433102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-30c11d15-0d80-4674-a9ac-0449b7606ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:24.445219 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 810/1635] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:24.446695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9971b-fa9b-4783-8dbc-495d06699db1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.447068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9971b-fa9b-4783-8dbc-495d06699db1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.447501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9971b-fa9b-4783-8dbc-495d06699db1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.453688 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 811/1636] 10.222.0.69 () {54 vars in 912 bytes} [Fri Jun 24 10:35:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:35:24.463549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84414c-f639-41fa-a85f-d4d499a71f50 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.463844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84414c-f639-41fa-a85f-d4d499a71f50 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.464472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84414c-f639-41fa-a85f-d4d499a71f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.479091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:24.519900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8a84414c-f639-41fa-a85f-d4d499a71f50 None None] MFA Rules not processed for user `29dd9bd37d4b4f46bc5d1043361eb070`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:24.521857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8a84414c-f639-41fa-a85f-d4d499a71f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:24.644770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 826/1637] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:24.646651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac168-39e0-4540-aec6-39c0a2d2aa94 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.647249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac168-39e0-4540-aec6-39c0a2d2aa94 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.647758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac168-39e0-4540-aec6-39c0a2d2aa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.659320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 812/1638] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:24.659923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84414c-f639-41fa-a85f-d4d499a71f50 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:24.693534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-53aac168-39e0-4540-aec6-39c0a2d2aa94 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:24.695062 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-53aac168-39e0-4540-aec6-39c0a2d2aa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:24.727350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84414c-f639-41fa-a85f-d4d499a71f50 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:24.731710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71a1f1-74de-40ee-91a1-26a6ddd93971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:24.732585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71a1f1-74de-40ee-91a1-26a6ddd93971 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.733130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71a1f1-74de-40ee-91a1-26a6ddd93971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.733569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71a1f1-74de-40ee-91a1-26a6ddd93971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.778138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 827/1639] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:24.780519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac168-39e0-4540-aec6-39c0a2d2aa94 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:24.798013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac168-39e0-4540-aec6-39c0a2d2aa94 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:24.815767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b80fc4-7865-4c74-b7f1-0a4def2b43e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:24.816561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b80fc4-7865-4c74-b7f1-0a4def2b43e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:24.817227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b80fc4-7865-4c74-b7f1-0a4def2b43e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:24.817612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b80fc4-7865-4c74-b7f1-0a4def2b43e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:24.839862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71a1f1-74de-40ee-91a1-26a6ddd93971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:24.841558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71a1f1-74de-40ee-91a1-26a6ddd93971 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:24.841921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b80fc4-7865-4c74-b7f1-0a4def2b43e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:24.843491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b80fc4-7865-4c74-b7f1-0a4def2b43e6 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:24.865687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 828/1640] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:24 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:24.868851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 813/1641] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:24.982210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b80fc4-7865-4c74-b7f1-0a4def2b43e6 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:24.993466 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b80fc4-7865-4c74-b7f1-0a4def2b43e6 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:25.002007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c79d6-0675-4f28-a4ae-52717965f99e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:25.004990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c79d6-0675-4f28-a4ae-52717965f99e service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:25.005842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c79d6-0675-4f28-a4ae-52717965f99e service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:25.005842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c79d6-0675-4f28-a4ae-52717965f99e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:25.027812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c79d6-0675-4f28-a4ae-52717965f99e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:25.029347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c79d6-0675-4f28-a4ae-52717965f99e service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:25.053149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 829/1642] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:25.119087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71a1f1-74de-40ee-91a1-26a6ddd93971 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:25.131169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71a1f1-74de-40ee-91a1-26a6ddd93971 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:25.137782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd5ab8-cc0a-4177-a868-275008d6c8f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:25.138924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd5ab8-cc0a-4177-a868-275008d6c8f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:25.139441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd5ab8-cc0a-4177-a868-275008d6c8f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:25.139862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd5ab8-cc0a-4177-a868-275008d6c8f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:25.166508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:25.172445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd5ab8-cc0a-4177-a868-275008d6c8f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:25.175130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd5ab8-cc0a-4177-a868-275008d6c8f6 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:25.190643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 814/1643] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:26.378582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c79d6-0675-4f28-a4ae-52717965f99e service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:26.399260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c79d6-0675-4f28-a4ae-52717965f99e service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:26.413263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df26d8-2882-4050-85e4-996d40fe39c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:26.413940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df26d8-2882-4050-85e4-996d40fe39c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:26.414229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df26d8-2882-4050-85e4-996d40fe39c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:26.414502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df26d8-2882-4050-85e4-996d40fe39c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:26.473909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd5ab8-cc0a-4177-a868-275008d6c8f6 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:26.617258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df26d8-2882-4050-85e4-996d40fe39c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:26.619178 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df26d8-2882-4050-85e4-996d40fe39c2 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:26.634997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 830/1644] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:26.660170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd5ab8-cc0a-4177-a868-275008d6c8f6 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:26.670938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:26.671848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:26.672131 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:26.672410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] PATH_INFO: `/v3/users/4c652ef9b3ea4f29aa68c0e8ed4c16f3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:26.674058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c652ef9b3ea4f29aa68c0e8ed4c16f3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:26.675119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:35:26.675119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 24 10:35:26.675119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: Jun 24 10:35:26.676384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:26.811402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c652ef9b3ea4f29aa68c0e8ed4c16f3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:26.855376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c652ef9b3ea4f29aa68c0e8ed4c16f3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:26.868381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c652ef9b3ea4f29aa68c0e8ed4c16f3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:26.906892 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c652ef9b3ea4f29aa68c0e8ed4c16f3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:26.908268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c652ef9b3ea4f29aa68c0e8ed4c16f3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:26.926466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 815/1645] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:26 2022] DELETE /identity/v3/users/4c652ef9b3ea4f29aa68c0e8ed4c16f3 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:35:26.930783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40180ecb-1042-4cd6-8267-a969977cc67f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:26.931109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40180ecb-1042-4cd6-8267-a969977cc67f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:26.931360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40180ecb-1042-4cd6-8267-a969977cc67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:26.965928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-40180ecb-1042-4cd6-8267-a969977cc67f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:26.967197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-40180ecb-1042-4cd6-8267-a969977cc67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:26.994334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:27.094870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 831/1646] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:27.112366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:27.114730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:27.199088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7761c-a1d2-4b1d-87b4-2361956abeea admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:27.204391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81d03d-add8-4b71-adfd-84f6ecdeb8e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:27.205214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81d03d-add8-4b71-adfd-84f6ecdeb8e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:27.205498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81d03d-add8-4b71-adfd-84f6ecdeb8e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:27.206129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81d03d-add8-4b71-adfd-84f6ecdeb8e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:27.221260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81d03d-add8-4b71-adfd-84f6ecdeb8e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:27.223496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81d03d-add8-4b71-adfd-84f6ecdeb8e5 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:27.240961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 816/1647] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:28.295269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40180ecb-1042-4cd6-8267-a969977cc67f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:28.297149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-40180ecb-1042-4cd6-8267-a969977cc67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:28.326050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40180ecb-1042-4cd6-8267-a969977cc67f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:28.340229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:28.341727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:28.342025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:28.342305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] PATH_INFO: `/v3/projects/2c0d7938e98147cc8dd73e1a89fd2c31` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:28.344259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0d7938e98147cc8dd73e1a89fd2c31)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:28.352534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:35:28.352534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Jun 24 10:35:28.352534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: Jun 24 10:35:28.353843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:28.545823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0d7938e98147cc8dd73e1a89fd2c31', 'request_id': 'req-d768019b-4a05-4e16-be54-77f272c7a7f1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:28.580619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0d7938e98147cc8dd73e1a89fd2c31', 'request_id': 'req-d768019b-4a05-4e16-be54-77f272c7a7f1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:28.693838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0d7938e98147cc8dd73e1a89fd2c31', 'request_id': 'req-d768019b-4a05-4e16-be54-77f272c7a7f1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:28.760473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] The token cache is being invalidate because project 2c0d7938e98147cc8dd73e1a89fd2c31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:35:28.766912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:28.768289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:28.779891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 832/1648] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:35:28 2022] DELETE /identity/v3/projects/2c0d7938e98147cc8dd73e1a89fd2c31 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:28.780616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81d03d-add8-4b71-adfd-84f6ecdeb8e5 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:28.792316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81d03d-add8-4b71-adfd-84f6ecdeb8e5 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:28.841327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:28.955642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:28.958837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:28.959792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:28.960321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] PATH_INFO: `/v3/users/e004515e2ef847a793cd77fbe1e29d75` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:28.966560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e004515e2ef847a793cd77fbe1e29d75)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:28.970249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:29.097770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e004515e2ef847a793cd77fbe1e29d75'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:29.134416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e004515e2ef847a793cd77fbe1e29d75'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:29.150370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e004515e2ef847a793cd77fbe1e29d75'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:29.170522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e004515e2ef847a793cd77fbe1e29d75'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:29.171489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bc57b071-10ec-4598-afff-9634da189dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e004515e2ef847a793cd77fbe1e29d75'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:29.183499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 817/1649] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:28 2022] DELETE /identity/v3/users/e004515e2ef847a793cd77fbe1e29d75 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:29.528060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:29.530165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:29.552016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768019b-4a05-4e16-be54-77f272c7a7f1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:29.697097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:29.698341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:29.698784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:29.699346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] PATH_INFO: `/v3/users/9bb081bd64964cabb5986eb7cb976617` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:29.706106 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb081bd64964cabb5986eb7cb976617)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:29.721244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:29.833517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:29.839950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb081bd64964cabb5986eb7cb976617'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:29.876204 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb081bd64964cabb5986eb7cb976617'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:29.935013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb081bd64964cabb5986eb7cb976617'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:29.936376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb081bd64964cabb5986eb7cb976617'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:29.996525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb081bd64964cabb5986eb7cb976617'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:30.027171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 833/1650] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:29 2022] DELETE /identity/v3/users/9bb081bd64964cabb5986eb7cb976617 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:30.031326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9ae02-8a00-4480-8dea-e4dabc810090 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:30.032177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9ae02-8a00-4480-8dea-e4dabc810090 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:30.032647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9ae02-8a00-4480-8dea-e4dabc810090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:30.079573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f6e9ae02-8a00-4480-8dea-e4dabc810090 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:30.096329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f6e9ae02-8a00-4480-8dea-e4dabc810090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:30.262376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 818/1651] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:30.465663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:30.466356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:30.614093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1ca2f-1455-48f5-8dc2-fbc05693ed32 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:30.626249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1e371-9b4d-4f09-a3ba-c0fe9db885c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:30.627532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1e371-9b4d-4f09-a3ba-c0fe9db885c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:30.628017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1e371-9b4d-4f09-a3ba-c0fe9db885c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:30.628478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1e371-9b4d-4f09-a3ba-c0fe9db885c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:30.651698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1e371-9b4d-4f09-a3ba-c0fe9db885c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:30.658671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1e371-9b4d-4f09-a3ba-c0fe9db885c5 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:30.683739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 834/1652] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:31.156568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9ae02-8a00-4480-8dea-e4dabc810090 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:31.158402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f6e9ae02-8a00-4480-8dea-e4dabc810090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:31.179599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9ae02-8a00-4480-8dea-e4dabc810090 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:31.190459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:31.191373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:31.191657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:31.192443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] PATH_INFO: `/v3/projects/26584a16924744169ca6ffd7fd2a8cda` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:31.195938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26584a16924744169ca6ffd7fd2a8cda)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:31.201944 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:31.296136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:31.317902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26584a16924744169ca6ffd7fd2a8cda', 'request_id': 'req-ff7cf322-76de-4c0c-a546-ff4be00cec59'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:31.331047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26584a16924744169ca6ffd7fd2a8cda', 'request_id': 'req-ff7cf322-76de-4c0c-a546-ff4be00cec59'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:31.346086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26584a16924744169ca6ffd7fd2a8cda', 'request_id': 'req-ff7cf322-76de-4c0c-a546-ff4be00cec59'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:31.364643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] The token cache is being invalidate because project 26584a16924744169ca6ffd7fd2a8cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:35:31.365219 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:31.366135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ff7cf322-76de-4c0c-a546-ff4be00cec59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:31.370510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 819/1653] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:35:31 2022] DELETE /identity/v3/projects/26584a16924744169ca6ffd7fd2a8cda => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:31.375567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242dfaa-039d-464e-aa36-69599de74490 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:31.375907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242dfaa-039d-464e-aa36-69599de74490 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:31.376554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242dfaa-039d-464e-aa36-69599de74490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:31.462811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7242dfaa-039d-464e-aa36-69599de74490 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:31.465376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7242dfaa-039d-464e-aa36-69599de74490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:31.597360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 835/1654] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:31.605079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e058c0-6922-49f9-b4d5-1020517d0849 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:31.605567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e058c0-6922-49f9-b4d5-1020517d0849 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:31.605913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e058c0-6922-49f9-b4d5-1020517d0849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:31.638927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-21e058c0-6922-49f9-b4d5-1020517d0849 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:31.645546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-21e058c0-6922-49f9-b4d5-1020517d0849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:31.659419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242dfaa-039d-464e-aa36-69599de74490 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:31.666110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 820/1655] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:31.673425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242dfaa-039d-464e-aa36-69599de74490 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:31.680998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cc49d-02f2-4ad9-b2da-46bb630ecd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:31.681876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cc49d-02f2-4ad9-b2da-46bb630ecd07 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:31.682315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cc49d-02f2-4ad9-b2da-46bb630ecd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:31.682758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cc49d-02f2-4ad9-b2da-46bb630ecd07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:31.683909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cc49d-02f2-4ad9-b2da-46bb630ecd07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:31.685354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cc49d-02f2-4ad9-b2da-46bb630ecd07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:31.687698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cc49d-02f2-4ad9-b2da-46bb630ecd07 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:31.734299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e058c0-6922-49f9-b4d5-1020517d0849 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:31.741058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 836/1656] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:31.747900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e058c0-6922-49f9-b4d5-1020517d0849 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:31.754913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de720cfd-3435-48bb-bcab-18e4b9a2f147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:31.755752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720cfd-3435-48bb-bcab-18e4b9a2f147 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:31.756037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720cfd-3435-48bb-bcab-18e4b9a2f147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:31.756424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720cfd-3435-48bb-bcab-18e4b9a2f147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:31.757207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de720cfd-3435-48bb-bcab-18e4b9a2f147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:31.759515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de720cfd-3435-48bb-bcab-18e4b9a2f147 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:31.820076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cc49d-02f2-4ad9-b2da-46bb630ecd07 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:31.822388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 821/1657] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:31 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:31.840488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cc49d-02f2-4ad9-b2da-46bb630ecd07 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:31.955293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:31.990001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a1580-ca3d-431f-8877-2791cce6b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:31.990829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a1580-ca3d-431f-8877-2791cce6b53e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:31.991192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a1580-ca3d-431f-8877-2791cce6b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:31.991574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a1580-ca3d-431f-8877-2791cce6b53e admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:31.992423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a1580-ca3d-431f-8877-2791cce6b53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:31.995446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a1580-ca3d-431f-8877-2791cce6b53e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:32.061523 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 837/1658] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:31 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:35:32.066009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de720cfd-3435-48bb-bcab-18e4b9a2f147 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:32.081826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de720cfd-3435-48bb-bcab-18e4b9a2f147 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:32.110783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035ead7-75d2-4bed-9e19-95a326820df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:32.111815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035ead7-75d2-4bed-9e19-95a326820df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:32.115080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035ead7-75d2-4bed-9e19-95a326820df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:32.116101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035ead7-75d2-4bed-9e19-95a326820df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:32.119510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035ead7-75d2-4bed-9e19-95a326820df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:32.139943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035ead7-75d2-4bed-9e19-95a326820df7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:32.173045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 822/1659] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:32.178555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a1580-ca3d-431f-8877-2791cce6b53e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:32.210830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a1580-ca3d-431f-8877-2791cce6b53e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:32.245718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbf3ef-166f-429b-a557-8a91014b57a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:32.252797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbf3ef-166f-429b-a557-8a91014b57a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:32.253358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbf3ef-166f-429b-a557-8a91014b57a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:32.256509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbf3ef-166f-429b-a557-8a91014b57a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:32.258036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbf3ef-166f-429b-a557-8a91014b57a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:32.259763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbf3ef-166f-429b-a557-8a91014b57a9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:32.287376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 838/1660] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:32.292164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035ead7-75d2-4bed-9e19-95a326820df7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:32.309815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035ead7-75d2-4bed-9e19-95a326820df7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:32.326262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f4cee-b1b8-4c73-9f18-8bad39183113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:32.327525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f4cee-b1b8-4c73-9f18-8bad39183113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:32.327899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f4cee-b1b8-4c73-9f18-8bad39183113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:32.328265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f4cee-b1b8-4c73-9f18-8bad39183113 admin admin] PATH_INFO: `/v3/projects/f219fe4790504a588953fa0756c6edd0/users/23aea071939d4e80bfc9680293dc518c/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:32.373288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f4cee-b1b8-4c73-9f18-8bad39183113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f219fe4790504a588953fa0756c6edd0, user_id=23aea071939d4e80bfc9680293dc518c, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:32.382006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f4cee-b1b8-4c73-9f18-8bad39183113 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:32.413773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 823/1661] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:32 2022] PUT /identity/v3/projects/f219fe4790504a588953fa0756c6edd0/users/23aea071939d4e80bfc9680293dc518c/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:32.426690 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746301a-1e0a-4968-98cf-3f32e1f85041 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:32.426988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746301a-1e0a-4968-98cf-3f32e1f85041 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:32.427269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746301a-1e0a-4968-98cf-3f32e1f85041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:32.495784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b746301a-1e0a-4968-98cf-3f32e1f85041 None None] MFA Rules not processed for user `23aea071939d4e80bfc9680293dc518c`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:32.497254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b746301a-1e0a-4968-98cf-3f32e1f85041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:32.607741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 839/1662] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:32.611943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f4cee-b1b8-4c73-9f18-8bad39183113 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:32.635770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd633ec7-da40-4fec-b76c-28e33ecfdc64 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:32.636079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd633ec7-da40-4fec-b76c-28e33ecfdc64 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:32.636347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd633ec7-da40-4fec-b76c-28e33ecfdc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:32.700343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cd633ec7-da40-4fec-b76c-28e33ecfdc64 None None] MFA Rules not processed for user `16daeab4accb4eeca4100b55dc9bcaa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:32.702768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cd633ec7-da40-4fec-b76c-28e33ecfdc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:32.780299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f4cee-b1b8-4c73-9f18-8bad39183113 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:32.787808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a57f81-fc18-4a00-ace9-3e712961956b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:32.789549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a57f81-fc18-4a00-ace9-3e712961956b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:32.790014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a57f81-fc18-4a00-ace9-3e712961956b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:32.790457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a57f81-fc18-4a00-ace9-3e712961956b admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:32.793326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a57f81-fc18-4a00-ace9-3e712961956b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:32.796218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a57f81-fc18-4a00-ace9-3e712961956b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:32.859556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 824/1663] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:32 2022] POST /identity/v3/projects => generated 371 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:32.859556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a57f81-fc18-4a00-ace9-3e712961956b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:32.861435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 840/1664] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:32.862345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd633ec7-da40-4fec-b76c-28e33ecfdc64 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:32.865172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-52a57f81-fc18-4a00-ace9-3e712961956b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:32.888453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd633ec7-da40-4fec-b76c-28e33ecfdc64 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:33.031390 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:33.050236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:33.101215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b26ece-c155-4e42-ab57-be2027c8ff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:33.101921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26ece-c155-4e42-ab57-be2027c8ff4a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:33.102213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26ece-c155-4e42-ab57-be2027c8ff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:33.102487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26ece-c155-4e42-ab57-be2027c8ff4a admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:33.103197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b26ece-c155-4e42-ab57-be2027c8ff4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:33.112266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b26ece-c155-4e42-ab57-be2027c8ff4a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:33.125772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a57f81-fc18-4a00-ace9-3e712961956b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:33.148314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:33.149871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:33.157025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:33.157410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] PATH_INFO: `/v3/projects/8539804dc27f42fe88833acb7b6cfa87` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:33.159072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8539804dc27f42fe88833acb7b6cfa87)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:33.161239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:33.182554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 841/1665] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:32 2022] POST /identity/v3/users => generated 292 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:33.187208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519efe76-6bfe-4dfc-bb67-4b48d60846c1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:33.187756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519efe76-6bfe-4dfc-bb67-4b48d60846c1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:33.190630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519efe76-6bfe-4dfc-bb67-4b48d60846c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:33.240873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-519efe76-6bfe-4dfc-bb67-4b48d60846c1 None None] MFA Rules not processed for user `16daeab4accb4eeca4100b55dc9bcaa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:33.242690 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-519efe76-6bfe-4dfc-bb67-4b48d60846c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:33.329381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8539804dc27f42fe88833acb7b6cfa87', 'request_id': 'req-f5d17695-057e-4da0-a7fb-e5dd99267ec3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:33.344680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8539804dc27f42fe88833acb7b6cfa87', 'request_id': 'req-f5d17695-057e-4da0-a7fb-e5dd99267ec3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:33.372664 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8539804dc27f42fe88833acb7b6cfa87', 'request_id': 'req-f5d17695-057e-4da0-a7fb-e5dd99267ec3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:33.401692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 842/1666] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:33.402854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519efe76-6bfe-4dfc-bb67-4b48d60846c1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:33.403587 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] The token cache is being invalidate because project 8539804dc27f42fe88833acb7b6cfa87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:35:33.403969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:33.404728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f5d17695-057e-4da0-a7fb-e5dd99267ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:33.406907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 825/1667] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:35:32 2022] DELETE /identity/v3/projects/8539804dc27f42fe88833acb7b6cfa87 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:33.412581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9b9f-8ac8-4dad-a8c1-fb91dcb1e64c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:33.412991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9b9f-8ac8-4dad-a8c1-fb91dcb1e64c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:33.413335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9b9f-8ac8-4dad-a8c1-fb91dcb1e64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:33.450808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519efe76-6bfe-4dfc-bb67-4b48d60846c1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:33.458306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b75a9b9f-8ac8-4dad-a8c1-fb91dcb1e64c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:33.463670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b75a9b9f-8ac8-4dad-a8c1-fb91dcb1e64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:33.644983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39ec0e-9482-45fc-b23b-738edc4d74b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:33.645978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39ec0e-9482-45fc-b23b-738edc4d74b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:33.646390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39ec0e-9482-45fc-b23b-738edc4d74b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:33.646797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39ec0e-9482-45fc-b23b-738edc4d74b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:33.647688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39ec0e-9482-45fc-b23b-738edc4d74b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:33.649652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39ec0e-9482-45fc-b23b-738edc4d74b3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:33.668472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 843/1668] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:33.669602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39ec0e-9482-45fc-b23b-738edc4d74b3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:33.673969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0b39ec0e-9482-45fc-b23b-738edc4d74b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:33.715461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 826/1669] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:35:33.715461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9b9f-8ac8-4dad-a8c1-fb91dcb1e64c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:33.716363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b75a9b9f-8ac8-4dad-a8c1-fb91dcb1e64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:33.742701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9b9f-8ac8-4dad-a8c1-fb91dcb1e64c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:33.751579 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac5d02-0ac5-4b64-9abc-7231dab858a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:33.758219 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac5d02-0ac5-4b64-9abc-7231dab858a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:33.760402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac5d02-0ac5-4b64-9abc-7231dab858a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:33.760803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac5d02-0ac5-4b64-9abc-7231dab858a6 admin admin] PATH_INFO: `/v3/projects/8f26550df6774d4c819f5e29d4a761ea/users/da1b665ca63d428f93fb2a1dbb5f02db/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:33.824420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac5d02-0ac5-4b64-9abc-7231dab858a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f26550df6774d4c819f5e29d4a761ea, user_id=da1b665ca63d428f93fb2a1dbb5f02db, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:33.831872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac5d02-0ac5-4b64-9abc-7231dab858a6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:33.877730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39ec0e-9482-45fc-b23b-738edc4d74b3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:33.883676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 827/1670] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:33 2022] PUT /identity/v3/projects/8f26550df6774d4c819f5e29d4a761ea/users/da1b665ca63d428f93fb2a1dbb5f02db/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:33.887863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d00212-4b7a-47ac-b882-c549f3e766c8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:33.888173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d00212-4b7a-47ac-b882-c549f3e766c8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:33.888446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d00212-4b7a-47ac-b882-c549f3e766c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:33.919097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:33.969188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e8d00212-4b7a-47ac-b882-c549f3e766c8 None None] MFA Rules not processed for user `733b7d5058da45089a2133e2fe4695e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:33.970983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e8d00212-4b7a-47ac-b882-c549f3e766c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:34.029088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:34.042562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab190951-fb9e-4401-9768-863bd368bd95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:34.042562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab190951-fb9e-4401-9768-863bd368bd95 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:34.042562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab190951-fb9e-4401-9768-863bd368bd95 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:34.042562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab190951-fb9e-4401-9768-863bd368bd95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:34.044588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ab190951-fb9e-4401-9768-863bd368bd95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:34.076160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 828/1671] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:34.083724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aae9b5-ed47-427d-ba0c-b5e92df06272 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:34.084623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aae9b5-ed47-427d-ba0c-b5e92df06272 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:34.085273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aae9b5-ed47-427d-ba0c-b5e92df06272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:34.109762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f1aae9b5-ed47-427d-ba0c-b5e92df06272 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:34.113479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f1aae9b5-ed47-427d-ba0c-b5e92df06272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:34.186457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab190951-fb9e-4401-9768-863bd368bd95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:34.212089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab190951-fb9e-4401-9768-863bd368bd95 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:34.276116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 844/1672] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:34.286860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 829/1673] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:34.288536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab190951-fb9e-4401-9768-863bd368bd95 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:34.290121 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aae9b5-ed47-427d-ba0c-b5e92df06272 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:34.291776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f1aae9b5-ed47-427d-ba0c-b5e92df06272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:34.321669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aae9b5-ed47-427d-ba0c-b5e92df06272 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:34.330503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f94ceb-9a6a-4614-996e-6898e3bf0b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:34.332209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f94ceb-9a6a-4614-996e-6898e3bf0b06 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:34.332689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f94ceb-9a6a-4614-996e-6898e3bf0b06 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:34.333203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f94ceb-9a6a-4614-996e-6898e3bf0b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:34.336484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab190951-fb9e-4401-9768-863bd368bd95 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:34.344125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f94ceb-9a6a-4614-996e-6898e3bf0b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:34.346605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f94ceb-9a6a-4614-996e-6898e3bf0b06 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:34.355509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035c946-b15a-45f2-ae54-0eb6d5f08a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:34.357314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035c946-b15a-45f2-ae54-0eb6d5f08a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:34.360487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035c946-b15a-45f2-ae54-0eb6d5f08a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:34.361748 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035c946-b15a-45f2-ae54-0eb6d5f08a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:34.362773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035c946-b15a-45f2-ae54-0eb6d5f08a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:34.374811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035c946-b15a-45f2-ae54-0eb6d5f08a2d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:34.378865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 830/1674] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:34.381047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f94ceb-9a6a-4614-996e-6898e3bf0b06 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:34.412251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 845/1675] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:34.413324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f94ceb-9a6a-4614-996e-6898e3bf0b06 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:34.417089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035c946-b15a-45f2-ae54-0eb6d5f08a2d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:34.437304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbf16f-692f-449f-bed0-ee8da8469c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:34.437762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbf16f-692f-449f-bed0-ee8da8469c9c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:34.438216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbf16f-692f-449f-bed0-ee8da8469c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:34.438509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbf16f-692f-449f-bed0-ee8da8469c9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:34.447409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbf16f-692f-449f-bed0-ee8da8469c9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:34.449252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbf16f-692f-449f-bed0-ee8da8469c9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:34.453421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035c946-b15a-45f2-ae54-0eb6d5f08a2d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:34.455383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbf16f-692f-449f-bed0-ee8da8469c9c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:34.466511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597707f-653a-41df-bc0c-66d69f8a5b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:34.467436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597707f-653a-41df-bc0c-66d69f8a5b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:34.467720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597707f-653a-41df-bc0c-66d69f8a5b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:34.467999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597707f-653a-41df-bc0c-66d69f8a5b3e admin admin] PATH_INFO: `/v3/projects/8f26550df6774d4c819f5e29d4a761ea/users/da1b665ca63d428f93fb2a1dbb5f02db/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:34.475997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597707f-653a-41df-bc0c-66d69f8a5b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f26550df6774d4c819f5e29d4a761ea, user_id=da1b665ca63d428f93fb2a1dbb5f02db, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:34.478784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597707f-653a-41df-bc0c-66d69f8a5b3e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:34.512927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 846/1676] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:34 2022] PUT /identity/v3/projects/8f26550df6774d4c819f5e29d4a761ea/users/da1b665ca63d428f93fb2a1dbb5f02db/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:34.514817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597707f-653a-41df-bc0c-66d69f8a5b3e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:34.516520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 831/1677] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:34.519779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43b50-f604-4d26-823d-886589df2a0d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:34.520074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43b50-f604-4d26-823d-886589df2a0d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:34.520349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43b50-f604-4d26-823d-886589df2a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:34.644047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-94f43b50-f604-4d26-823d-886589df2a0d None None] MFA Rules not processed for user `da1b665ca63d428f93fb2a1dbb5f02db`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:34.650987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-94f43b50-f604-4d26-823d-886589df2a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:34.692059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597707f-653a-41df-bc0c-66d69f8a5b3e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:34.709993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abea655-8b5b-4645-b13a-d6ae6c67a1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:34.711051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abea655-8b5b-4645-b13a-d6ae6c67a1cd admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:34.711648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abea655-8b5b-4645-b13a-d6ae6c67a1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:34.712082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abea655-8b5b-4645-b13a-d6ae6c67a1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:34.714003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abea655-8b5b-4645-b13a-d6ae6c67a1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:34.717792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abea655-8b5b-4645-b13a-d6ae6c67a1cd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:34.799722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 847/1678] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:34 2022] POST /identity/v3/projects => generated 401 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:34.800717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abea655-8b5b-4645-b13a-d6ae6c67a1cd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:34.832851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:34.888063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 832/1679] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:34.902507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f43b50-f604-4d26-823d-886589df2a0d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:34.903943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-94f43b50-f604-4d26-823d-886589df2a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:34.941220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abea655-8b5b-4645-b13a-d6ae6c67a1cd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:34.971794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783c4d5-98c3-4723-9de6-b453f834c0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:34.972514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783c4d5-98c3-4723-9de6-b453f834c0ee admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:34.972819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783c4d5-98c3-4723-9de6-b453f834c0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:34.973088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783c4d5-98c3-4723-9de6-b453f834c0ee admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:34.974576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783c4d5-98c3-4723-9de6-b453f834c0ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:34.976719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783c4d5-98c3-4723-9de6-b453f834c0ee admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:35.014638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:35.061521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 848/1680] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:34 2022] POST /identity/v3/users => generated 301 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:35.066128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783c4d5-98c3-4723-9de6-b453f834c0ee admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:35.091514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f43b50-f604-4d26-823d-886589df2a0d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:35.112233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783c4d5-98c3-4723-9de6-b453f834c0ee admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:35.120509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ab0f3-4c8d-45db-af53-a00d417b2e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:35.121230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ab0f3-4c8d-45db-af53-a00d417b2e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.121515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ab0f3-4c8d-45db-af53-a00d417b2e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.121792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ab0f3-4c8d-45db-af53-a00d417b2e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.126837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ab0f3-4c8d-45db-af53-a00d417b2e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:35.128529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ab0f3-4c8d-45db-af53-a00d417b2e27 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:35.168431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 849/1681] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:35.169630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ab0f3-4c8d-45db-af53-a00d417b2e27 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:35.186924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ab0f3-4c8d-45db-af53-a00d417b2e27 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:35.201479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607ea64-8015-4bed-9a33-bc13485e098f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:35.202022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607ea64-8015-4bed-9a33-bc13485e098f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.202314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607ea64-8015-4bed-9a33-bc13485e098f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.202593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607ea64-8015-4bed-9a33-bc13485e098f admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.203335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607ea64-8015-4bed-9a33-bc13485e098f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:35.210678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607ea64-8015-4bed-9a33-bc13485e098f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:35.211344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200cf80-4d1a-4ec6-aee5-727cd3271005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:35.212067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200cf80-4d1a-4ec6-aee5-727cd3271005 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.212304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200cf80-4d1a-4ec6-aee5-727cd3271005 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.212580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200cf80-4d1a-4ec6-aee5-727cd3271005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.245043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 850/1682] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:35.246056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607ea64-8015-4bed-9a33-bc13485e098f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:35.265805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607ea64-8015-4bed-9a33-bc13485e098f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:35.288963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eba4e-524d-42a3-9d2d-8625c7819a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:35.289891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eba4e-524d-42a3-9d2d-8625c7819a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.290175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eba4e-524d-42a3-9d2d-8625c7819a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.297157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eba4e-524d-42a3-9d2d-8625c7819a1e admin admin] PATH_INFO: `/v3/projects/37f6417cf2ec4f84aa826133fcece108/users/51c5d934c4e641578f9fb7a72731584f/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.337036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9eba4e-524d-42a3-9d2d-8625c7819a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f6417cf2ec4f84aa826133fcece108, user_id=51c5d934c4e641578f9fb7a72731584f, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:35.339690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200cf80-4d1a-4ec6-aee5-727cd3271005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:35.341186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200cf80-4d1a-4ec6-aee5-727cd3271005 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:35.345165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9eba4e-524d-42a3-9d2d-8625c7819a1e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:35.375003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 851/1683] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:35 2022] PUT /identity/v3/projects/37f6417cf2ec4f84aa826133fcece108/users/51c5d934c4e641578f9fb7a72731584f/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:35.382386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e19f5-1f5f-40ce-8871-5802ece374a2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.382681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e19f5-1f5f-40ce-8871-5802ece374a2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.382961 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e19f5-1f5f-40ce-8871-5802ece374a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.385988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 833/1684] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:35:35.437954 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ec6e19f5-1f5f-40ce-8871-5802ece374a2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:35.444862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ec6e19f5-1f5f-40ce-8871-5802ece374a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:35.526915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200cf80-4d1a-4ec6-aee5-727cd3271005 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:35.528567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c200cf80-4d1a-4ec6-aee5-727cd3271005 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:35.552609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200cf80-4d1a-4ec6-aee5-727cd3271005 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:35.576705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 852/1685] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:35.578594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec8628-9815-4909-9a30-5240e6ab7e77 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.578594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec8628-9815-4909-9a30-5240e6ab7e77 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.579047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec8628-9815-4909-9a30-5240e6ab7e77 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.580465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 853/1686] 10.222.0.69 () {54 vars in 912 bytes} [Fri Jun 24 10:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:35:35.587709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbd927-6946-442c-825e-3bd197274696 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.588091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbd927-6946-442c-825e-3bd197274696 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.588783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbd927-6946-442c-825e-3bd197274696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.673875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8bcbd927-6946-442c-825e-3bd197274696 None None] MFA Rules not processed for user `a86de22b0e324c70aad792ec6b1a7e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:35.674922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f383914-9a86-4767-b009-7eada531db7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:35.675129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8bcbd927-6946-442c-825e-3bd197274696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:35.676821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f383914-9a86-4767-b009-7eada531db7a service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.677851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f383914-9a86-4767-b009-7eada531db7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.683375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f383914-9a86-4767-b009-7eada531db7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.688425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3f383914-9a86-4767-b009-7eada531db7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:35.788637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:35.823462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 854/1687] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:35:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:35:35.824432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbd927-6946-442c-825e-3bd197274696 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:35.834060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8bcbd927-6946-442c-825e-3bd197274696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:35.865850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbd927-6946-442c-825e-3bd197274696 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:35.876431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4b793-4c36-4ca6-a5ca-086cb858be4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:35.879183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4b793-4c36-4ca6-a5ca-086cb858be4c service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.879732 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4b793-4c36-4ca6-a5ca-086cb858be4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.880210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4b793-4c36-4ca6-a5ca-086cb858be4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.909774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4b793-4c36-4ca6-a5ca-086cb858be4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:35.911998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4b793-4c36-4ca6-a5ca-086cb858be4c service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:35.913255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f383914-9a86-4767-b009-7eada531db7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:35.916300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f383914-9a86-4767-b009-7eada531db7a service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:35.926827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:35.953269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 855/1688] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:35.955494 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 834/1689] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:35:35.955601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4b793-4c36-4ca6-a5ca-086cb858be4c service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:35.968810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4b793-4c36-4ca6-a5ca-086cb858be4c service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:35.973726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef4daf-238e-40d3-bdaf-285182a2d8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:35.974262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef4daf-238e-40d3-bdaf-285182a2d8df service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:35.974646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef4daf-238e-40d3-bdaf-285182a2d8df service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:35.975021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef4daf-238e-40d3-bdaf-285182a2d8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:35.984078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef4daf-238e-40d3-bdaf-285182a2d8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:35.985889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef4daf-238e-40d3-bdaf-285182a2d8df service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:36.011245 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 856/1690] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:36.978253 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffed198-e362-496a-a3a5-70be6f527e8b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:36.978791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffed198-e362-496a-a3a5-70be6f527e8b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:36.978874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffed198-e362-496a-a3a5-70be6f527e8b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:36.985747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 835/1691] 10.222.0.69 () {54 vars in 960 bytes} [Fri Jun 24 10:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:35:36.987103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cb615-1ff4-4bb7-a018-0d9b0f50f1c3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:36.987570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cb615-1ff4-4bb7-a018-0d9b0f50f1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:36.987758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cb615-1ff4-4bb7-a018-0d9b0f50f1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:37.032580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-522cb615-1ff4-4bb7-a018-0d9b0f50f1c3 None None] MFA Rules not processed for user `9373333a1fd64852a138b3f5913168c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:37.034959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-522cb615-1ff4-4bb7-a018-0d9b0f50f1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:37.212192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 857/1692] 10.222.0.69 () {58 vars in 1057 bytes} [Fri Jun 24 10:35:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:37.220881 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffed198-e362-496a-a3a5-70be6f527e8b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:37.260164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffed198-e362-496a-a3a5-70be6f527e8b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:37.287719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5c882-6f5a-43a8-bb4f-78bd1d5d77f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:37.289295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5c882-6f5a-43a8-bb4f-78bd1d5d77f3 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:37.290735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5c882-6f5a-43a8-bb4f-78bd1d5d77f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:37.291129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5c882-6f5a-43a8-bb4f-78bd1d5d77f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:37.300050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a1c5c882-6f5a-43a8-bb4f-78bd1d5d77f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:37.384888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5c882-6f5a-43a8-bb4f-78bd1d5d77f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:37.390090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5c882-6f5a-43a8-bb4f-78bd1d5d77f3 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:37.441248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 836/1693] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:37.650185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cb615-1ff4-4bb7-a018-0d9b0f50f1c3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:37.652326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-522cb615-1ff4-4bb7-a018-0d9b0f50f1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:37.710098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cb615-1ff4-4bb7-a018-0d9b0f50f1c3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:37.727810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:37.731121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:37.731426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:37.731710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] PATH_INFO: `/v3/users/b1a6ca70bb944325a9e9424853a0bc48` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:37.734345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a6ca70bb944325a9e9424853a0bc48)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:37.740534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:37.894597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a6ca70bb944325a9e9424853a0bc48'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:37.948465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a6ca70bb944325a9e9424853a0bc48'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:37.975416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a6ca70bb944325a9e9424853a0bc48'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:37.979721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a6ca70bb944325a9e9424853a0bc48'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:38.000503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a96ebf96-b0e5-4656-8e39-4d4d8d1f5222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a6ca70bb944325a9e9424853a0bc48'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:38.016809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 858/1694] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:37 2022] DELETE /identity/v3/users/b1a6ca70bb944325a9e9424853a0bc48 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:38.019003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91001183-f935-4ff0-8b0c-9229602ada94 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:38.019550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91001183-f935-4ff0-8b0c-9229602ada94 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:38.019733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91001183-f935-4ff0-8b0c-9229602ada94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:38.079247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-91001183-f935-4ff0-8b0c-9229602ada94 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:38.093103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-91001183-f935-4ff0-8b0c-9229602ada94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:38.289924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717be8a-aece-40d8-8f82-ab85ed9d2f77 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:38.290334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717be8a-aece-40d8-8f82-ab85ed9d2f77 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:38.290531 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717be8a-aece-40d8-8f82-ab85ed9d2f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:38.343616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7717be8a-aece-40d8-8f82-ab85ed9d2f77 None None] MFA Rules not processed for user `51c5d934c4e641578f9fb7a72731584f`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:38.346928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7717be8a-aece-40d8-8f82-ab85ed9d2f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:38.374398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 837/1695] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:38.389258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91001183-f935-4ff0-8b0c-9229602ada94 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:38.390925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-91001183-f935-4ff0-8b0c-9229602ada94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:38.415357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:38.541167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 859/1696] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:38.545201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717be8a-aece-40d8-8f82-ab85ed9d2f77 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:38.546726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7717be8a-aece-40d8-8f82-ab85ed9d2f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:38.566359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91001183-f935-4ff0-8b0c-9229602ada94 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:38.580349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc61a4c-33ac-4322-9438-ded50bd45438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:38.581325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc61a4c-33ac-4322-9438-ded50bd45438 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:38.581887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc61a4c-33ac-4322-9438-ded50bd45438 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:38.582028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717be8a-aece-40d8-8f82-ab85ed9d2f77 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:38.582442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc61a4c-33ac-4322-9438-ded50bd45438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:38.595291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea252462-7c83-4260-9241-387a588aa232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:38.596211 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea252462-7c83-4260-9241-387a588aa232 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:38.597044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea252462-7c83-4260-9241-387a588aa232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:38.597767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea252462-7c83-4260-9241-387a588aa232 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:38.598138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea252462-7c83-4260-9241-387a588aa232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:38.602653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea252462-7c83-4260-9241-387a588aa232 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:38.603692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc61a4c-33ac-4322-9438-ded50bd45438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:38.605516 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc61a4c-33ac-4322-9438-ded50bd45438 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:38.641066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 838/1697] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:38.675028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 860/1698] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:38 2022] POST /identity/v3/projects => generated 401 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:35:38.679636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc61a4c-33ac-4322-9438-ded50bd45438 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:38.694492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc61a4c-33ac-4322-9438-ded50bd45438 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:38.798943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e11cc-9b81-43e7-8be9-56430e67814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:38.799793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e11cc-9b81-43e7-8be9-56430e67814f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:38.800230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e11cc-9b81-43e7-8be9-56430e67814f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:38.800692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e11cc-9b81-43e7-8be9-56430e67814f admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:38.801577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e11cc-9b81-43e7-8be9-56430e67814f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:38.803989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e11cc-9b81-43e7-8be9-56430e67814f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:38.832524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 839/1699] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:38 2022] POST /identity/v3/users => generated 307 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:38.836452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea252462-7c83-4260-9241-387a588aa232 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:38.852425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea252462-7c83-4260-9241-387a588aa232 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:38.866564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cd799-8422-49cd-ac77-af3d0955c300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:38.867270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cd799-8422-49cd-ac77-af3d0955c300 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:38.868590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cd799-8422-49cd-ac77-af3d0955c300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:38.868931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cd799-8422-49cd-ac77-af3d0955c300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:38.872080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cd799-8422-49cd-ac77-af3d0955c300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:38.874457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cd799-8422-49cd-ac77-af3d0955c300 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:38.903057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 861/1700] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:38.905039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e11cc-9b81-43e7-8be9-56430e67814f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:38.920372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e11cc-9b81-43e7-8be9-56430e67814f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:38.937007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d217dd8-27b3-446a-a2ae-28501f39dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:38.938595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d217dd8-27b3-446a-a2ae-28501f39dbab admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:38.939944 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d217dd8-27b3-446a-a2ae-28501f39dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:38.939944 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d217dd8-27b3-446a-a2ae-28501f39dbab admin admin] PATH_INFO: `/v3/projects/9b9fecd90abb447396a888e466273e92/users/6538f5a08e354ae6aabf39d8f37d2782/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:38.958625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d217dd8-27b3-446a-a2ae-28501f39dbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9fecd90abb447396a888e466273e92, user_id=6538f5a08e354ae6aabf39d8f37d2782, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:38.964600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d217dd8-27b3-446a-a2ae-28501f39dbab admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:38.967620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:39.002531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 840/1701] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:38 2022] PUT /identity/v3/projects/9b9fecd90abb447396a888e466273e92/users/6538f5a08e354ae6aabf39d8f37d2782/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:39.003901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cd799-8422-49cd-ac77-af3d0955c300 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:39.017230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cd799-8422-49cd-ac77-af3d0955c300 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:39.161414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d217dd8-27b3-446a-a2ae-28501f39dbab admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:39.163450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8d217dd8-27b3-446a-a2ae-28501f39dbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:39.180997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c55e9-aca2-469c-830d-8e14a838daf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:39.181988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c55e9-aca2-469c-830d-8e14a838daf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:39.182408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c55e9-aca2-469c-830d-8e14a838daf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:39.182818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c55e9-aca2-469c-830d-8e14a838daf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:39.183723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c55e9-aca2-469c-830d-8e14a838daf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:39.185650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c55e9-aca2-469c-830d-8e14a838daf9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:39.200601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 862/1702] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:39.204935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c55e9-aca2-469c-830d-8e14a838daf9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:39.220074 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c55e9-aca2-469c-830d-8e14a838daf9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:39.228619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24821b5-15c8-449f-ac4a-4b4b8c53e939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:39.230163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24821b5-15c8-449f-ac4a-4b4b8c53e939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:39.231184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24821b5-15c8-449f-ac4a-4b4b8c53e939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:39.232001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24821b5-15c8-449f-ac4a-4b4b8c53e939 admin admin] PATH_INFO: `/v3/projects/9b9fecd90abb447396a888e466273e92/users/6538f5a08e354ae6aabf39d8f37d2782/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:39.235640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24821b5-15c8-449f-ac4a-4b4b8c53e939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9fecd90abb447396a888e466273e92, user_id=6538f5a08e354ae6aabf39d8f37d2782, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:39.240959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24821b5-15c8-449f-ac4a-4b4b8c53e939 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:39.257863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 863/1703] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:39 2022] PUT /identity/v3/projects/9b9fecd90abb447396a888e466273e92/users/6538f5a08e354ae6aabf39d8f37d2782/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:35:39.369564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d217dd8-27b3-446a-a2ae-28501f39dbab admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:39.598961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:39.600882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:39.601111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:39.601388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] PATH_INFO: `/v3/projects/98f3e7ee3a4648fcb630682d1ab59dba` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:39.606470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f3e7ee3a4648fcb630682d1ab59dba)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:39.612387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:39.727056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24821b5-15c8-449f-ac4a-4b4b8c53e939 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:39.731440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b24821b5-15c8-449f-ac4a-4b4b8c53e939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:39.822535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24821b5-15c8-449f-ac4a-4b4b8c53e939 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:39.850543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:39.853523 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:39.856470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:39.856991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] PATH_INFO: `/v3/users/d5354b3fb3664800b2f247a34d3de386` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:39.897754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f3e7ee3a4648fcb630682d1ab59dba', 'request_id': 'req-292cdfe6-742b-4030-8255-732736c00288'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:39.898695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5354b3fb3664800b2f247a34d3de386)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:39.909730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f3e7ee3a4648fcb630682d1ab59dba', 'request_id': 'req-292cdfe6-742b-4030-8255-732736c00288'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:39.916254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:39.934634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f3e7ee3a4648fcb630682d1ab59dba', 'request_id': 'req-292cdfe6-742b-4030-8255-732736c00288'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:39.954228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] The token cache is being invalidate because project 98f3e7ee3a4648fcb630682d1ab59dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:35:39.954886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:39.956520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-292cdfe6-742b-4030-8255-732736c00288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:39.972348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 841/1704] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:35:39 2022] DELETE /identity/v3/projects/98f3e7ee3a4648fcb630682d1ab59dba => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:39.984129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d160e2-a064-4e2b-b834-b937d59dc104 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:39.984729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d160e2-a064-4e2b-b834-b937d59dc104 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:39.985338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d160e2-a064-4e2b-b834-b937d59dc104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:40.007966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-34d160e2-a064-4e2b-b834-b937d59dc104 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:40.009349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34d160e2-a064-4e2b-b834-b937d59dc104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:40.102605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:40.118572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5354b3fb3664800b2f247a34d3de386'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:40.161141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5354b3fb3664800b2f247a34d3de386'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:40.182207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5354b3fb3664800b2f247a34d3de386'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:40.183145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5354b3fb3664800b2f247a34d3de386'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:40.185466 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:40.217029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 842/1705] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:40.222214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b0637-8fdc-42bd-89df-8577f65451d1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:40.222696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b0637-8fdc-42bd-89df-8577f65451d1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:40.223191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b0637-8fdc-42bd-89df-8577f65451d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:40.227880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7e32843a-57a8-4dcf-b53b-6906035a4acc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5354b3fb3664800b2f247a34d3de386'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:40.242967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 864/1706] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:39 2022] DELETE /identity/v3/users/d5354b3fb3664800b2f247a34d3de386 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:40.248270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd617c1-1188-4d3c-95dd-39c06f3b6b74 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:40.248683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd617c1-1188-4d3c-95dd-39c06f3b6b74 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:40.249116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd617c1-1188-4d3c-95dd-39c06f3b6b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:40.298670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8a5b0637-8fdc-42bd-89df-8577f65451d1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:40.300700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8a5b0637-8fdc-42bd-89df-8577f65451d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:40.306723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7fd617c1-1188-4d3c-95dd-39c06f3b6b74 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:40.311178 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7fd617c1-1188-4d3c-95dd-39c06f3b6b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:40.510121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 865/1707] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:40.541554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd617c1-1188-4d3c-95dd-39c06f3b6b74 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:40.543057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7fd617c1-1188-4d3c-95dd-39c06f3b6b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:40.564562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 843/1708] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:40.567640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b0637-8fdc-42bd-89df-8577f65451d1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:40.595828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b0637-8fdc-42bd-89df-8577f65451d1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:40.612057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ff094-24a4-4460-9cca-a1c5263f5d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:40.612570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ff094-24a4-4460-9cca-a1c5263f5d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:40.612888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ff094-24a4-4460-9cca-a1c5263f5d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:40.613164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ff094-24a4-4460-9cca-a1c5263f5d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:40.613999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ff094-24a4-4460-9cca-a1c5263f5d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:40.614339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ff094-24a4-4460-9cca-a1c5263f5d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:40.615988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ff094-24a4-4460-9cca-a1c5263f5d19 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:40.651876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 844/1709] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:40.653143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ff094-24a4-4460-9cca-a1c5263f5d19 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:40.664240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ff094-24a4-4460-9cca-a1c5263f5d19 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:40.674392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f63948-8ad4-44bf-97df-105e3125be7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:40.675104 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f63948-8ad4-44bf-97df-105e3125be7e admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:40.675386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f63948-8ad4-44bf-97df-105e3125be7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:40.675724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f63948-8ad4-44bf-97df-105e3125be7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:40.689041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f63948-8ad4-44bf-97df-105e3125be7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:40.689213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd617c1-1188-4d3c-95dd-39c06f3b6b74 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:40.691206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f63948-8ad4-44bf-97df-105e3125be7e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:40.705826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca76de-ac87-4c5d-9652-8ff381598860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:40.706572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca76de-ac87-4c5d-9652-8ff381598860 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:40.706859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca76de-ac87-4c5d-9652-8ff381598860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:40.707146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca76de-ac87-4c5d-9652-8ff381598860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:40.742939 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca76de-ac87-4c5d-9652-8ff381598860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:40.744983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca76de-ac87-4c5d-9652-8ff381598860 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:40.757464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 845/1710] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:40 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:40.762266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f63948-8ad4-44bf-97df-105e3125be7e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:40.762641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 866/1711] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:35:40.845759 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f63948-8ad4-44bf-97df-105e3125be7e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:40.852488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9d937-ec57-4f23-883b-81ce77b67a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:40.853505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9d937-ec57-4f23-883b-81ce77b67a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:40.854097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9d937-ec57-4f23-883b-81ce77b67a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:40.854533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9d937-ec57-4f23-883b-81ce77b67a3b admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:40.855383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9d937-ec57-4f23-883b-81ce77b67a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:40.858022 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9d937-ec57-4f23-883b-81ce77b67a3b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:40.893172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 846/1712] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:40 2022] POST /identity/v3/users => generated 283 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:40.893931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca76de-ac87-4c5d-9652-8ff381598860 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:40.906808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca76de-ac87-4c5d-9652-8ff381598860 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:40.924998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7d3e9-89e2-4908-b04d-1675d4b390d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:40.926355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7d3e9-89e2-4908-b04d-1675d4b390d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:40.926648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7d3e9-89e2-4908-b04d-1675d4b390d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:40.926927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7d3e9-89e2-4908-b04d-1675d4b390d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:40.928947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7d3e9-89e2-4908-b04d-1675d4b390d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:40.933483 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7d3e9-89e2-4908-b04d-1675d4b390d6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:40.958169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 867/1713] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:40.962160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9d937-ec57-4f23-883b-81ce77b67a3b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:40.980314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9d937-ec57-4f23-883b-81ce77b67a3b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:40.990362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae15655-e186-4a33-987c-02ea9b96689c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:40.991125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15655-e186-4a33-987c-02ea9b96689c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:40.991558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15655-e186-4a33-987c-02ea9b96689c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:40.991928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15655-e186-4a33-987c-02ea9b96689c admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:40.992602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae15655-e186-4a33-987c-02ea9b96689c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:41.004303 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae15655-e186-4a33-987c-02ea9b96689c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:41.035562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 847/1714] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:41.037334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7d3e9-89e2-4908-b04d-1675d4b390d6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:41.051150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7d3e9-89e2-4908-b04d-1675d4b390d6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:41.058256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464ab4e-9a13-49b7-95a0-c9c51cc974d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:41.059476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464ab4e-9a13-49b7-95a0-c9c51cc974d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:41.059969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464ab4e-9a13-49b7-95a0-c9c51cc974d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:41.060464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464ab4e-9a13-49b7-95a0-c9c51cc974d8 admin admin] PATH_INFO: `/v3/projects/021f1bf5e16b457487b35b5a3eee6441/users/79b4055ead3049e4a14892397c829c78/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:41.088580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464ab4e-9a13-49b7-95a0-c9c51cc974d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=021f1bf5e16b457487b35b5a3eee6441, user_id=79b4055ead3049e4a14892397c829c78, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:41.096405 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464ab4e-9a13-49b7-95a0-c9c51cc974d8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:41.118453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 868/1715] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:41 2022] PUT /identity/v3/projects/021f1bf5e16b457487b35b5a3eee6441/users/79b4055ead3049e4a14892397c829c78/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:41.129421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05071da1-fccc-4629-a12a-20d68ab3744f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:41.129668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05071da1-fccc-4629-a12a-20d68ab3744f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:41.129917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05071da1-fccc-4629-a12a-20d68ab3744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:41.168393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-05071da1-fccc-4629-a12a-20d68ab3744f None None] MFA Rules not processed for user `79b4055ead3049e4a14892397c829c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:41.170896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-05071da1-fccc-4629-a12a-20d68ab3744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:41.176950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:41.201589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5156fd-87e9-4e3c-9122-ab2d71408826 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:41.202129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5156fd-87e9-4e3c-9122-ab2d71408826 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:41.202129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5156fd-87e9-4e3c-9122-ab2d71408826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:41.259169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3f5156fd-87e9-4e3c-9122-ab2d71408826 None None] MFA Rules not processed for user `6538f5a08e354ae6aabf39d8f37d2782`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:41.261455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f5156fd-87e9-4e3c-9122-ab2d71408826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:41.265885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 848/1716] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:41.280212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05071da1-fccc-4629-a12a-20d68ab3744f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:41.282652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-05071da1-fccc-4629-a12a-20d68ab3744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:41.353960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:41.426408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 869/1717] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:41 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:35:41.471586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05071da1-fccc-4629-a12a-20d68ab3744f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:41.490309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648847f6-0d2b-44f6-aeb9-bc19f1527a18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:41.491018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648847f6-0d2b-44f6-aeb9-bc19f1527a18 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:41.491457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648847f6-0d2b-44f6-aeb9-bc19f1527a18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:41.491740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648847f6-0d2b-44f6-aeb9-bc19f1527a18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:41.516265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648847f6-0d2b-44f6-aeb9-bc19f1527a18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:41.518495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648847f6-0d2b-44f6-aeb9-bc19f1527a18 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:41.542099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 849/1718] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:42.164574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5156fd-87e9-4e3c-9122-ab2d71408826 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:42.167780 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f5156fd-87e9-4e3c-9122-ab2d71408826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:42.297592 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5156fd-87e9-4e3c-9122-ab2d71408826 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:42.310705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:42.312445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:42.312924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:42.313347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] PATH_INFO: `/v3/projects/dcbf3094963e41b9bb17026f4eeca3ca` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:42.314781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbf3094963e41b9bb17026f4eeca3ca)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:42.317022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:42.393260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbf3094963e41b9bb17026f4eeca3ca', 'request_id': 'req-113a63ae-1449-4a80-83d9-f3569093afc7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.413772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbf3094963e41b9bb17026f4eeca3ca', 'request_id': 'req-113a63ae-1449-4a80-83d9-f3569093afc7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.433740 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbf3094963e41b9bb17026f4eeca3ca', 'request_id': 'req-113a63ae-1449-4a80-83d9-f3569093afc7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.448258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] The token cache is being invalidate because project dcbf3094963e41b9bb17026f4eeca3ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:35:42.448966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.450043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-113a63ae-1449-4a80-83d9-f3569093afc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.456781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 870/1719] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:35:42 2022] DELETE /identity/v3/projects/dcbf3094963e41b9bb17026f4eeca3ca => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:42.457733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648847f6-0d2b-44f6-aeb9-bc19f1527a18 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:42.473723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648847f6-0d2b-44f6-aeb9-bc19f1527a18 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:42.598467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:42.599471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:42.599836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:42.600182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] PATH_INFO: `/v3/users/29dd9bd37d4b4f46bc5d1043361eb070` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:42.603402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=29dd9bd37d4b4f46bc5d1043361eb070)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:42.606858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:42.679429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f63dd2-c60e-4590-9a61-d823e7a8b06f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:42.679725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f63dd2-c60e-4590-9a61-d823e7a8b06f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:42.680138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f63dd2-c60e-4590-9a61-d823e7a8b06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:42.702867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29dd9bd37d4b4f46bc5d1043361eb070'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.733056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:42.754088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-91f63dd2-c60e-4590-9a61-d823e7a8b06f None None] MFA Rules not processed for user `3e4ed08c1c81453abaa6c1dd826f0258`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:42.756159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29dd9bd37d4b4f46bc5d1043361eb070'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.759365 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-91f63dd2-c60e-4590-9a61-d823e7a8b06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:42.771380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29dd9bd37d4b4f46bc5d1043361eb070'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.800683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29dd9bd37d4b4f46bc5d1043361eb070'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.802815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3abace38-b827-4a8f-9408-86918f0c89dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29dd9bd37d4b4f46bc5d1043361eb070'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:42.831419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 850/1720] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:42 2022] DELETE /identity/v3/users/29dd9bd37d4b4f46bc5d1043361eb070 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:42.835566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb225b7-d975-480b-aed7-a8c01a110549 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:42.835867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb225b7-d975-480b-aed7-a8c01a110549 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:42.836136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb225b7-d975-480b-aed7-a8c01a110549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:42.942195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-abb225b7-d975-480b-aed7-a8c01a110549 None None] MFA Rules not processed for user `960b18fe180f4185b291da1d9c18a2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:42.943436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-abb225b7-d975-480b-aed7-a8c01a110549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:43.006996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 871/1721] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:43.009145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8254417-5ce1-4913-8ea6-13e4023eceff None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:43.009433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8254417-5ce1-4913-8ea6-13e4023eceff None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:43.014936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8254417-5ce1-4913-8ea6-13e4023eceff None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:43.017903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 872/1722] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:35:43 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:35:43.023585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8254417-5ce1-4913-8ea6-13e4023eceff None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:43.033683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b8254417-5ce1-4913-8ea6-13e4023eceff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:43.064921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:43.241797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 851/1723] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:43.243354 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b65b88-bfcb-46e2-a8c0-ecf9623dea4d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:43.243593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b65b88-bfcb-46e2-a8c0-ecf9623dea4d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:43.243889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b65b88-bfcb-46e2-a8c0-ecf9623dea4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:43.308770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-06b65b88-bfcb-46e2-a8c0-ecf9623dea4d None None] MFA Rules not processed for user `733b7d5058da45089a2133e2fe4695e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:43.310261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-06b65b88-bfcb-46e2-a8c0-ecf9623dea4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:43.470821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 852/1724] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:43.472752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b105a-f421-488b-9f2f-e54c725603ae None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:43.473004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b105a-f421-488b-9f2f-e54c725603ae None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:43.473279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b105a-f421-488b-9f2f-e54c725603ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:43.480052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8254417-5ce1-4913-8ea6-13e4023eceff None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:43.501487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e8b5b-f779-4554-b1ad-ea6a1fac8893 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:43.502692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e8b5b-f779-4554-b1ad-ea6a1fac8893 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:43.503024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e8b5b-f779-4554-b1ad-ea6a1fac8893 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:43.503310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e8b5b-f779-4554-b1ad-ea6a1fac8893 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:43.512369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b55b105a-f421-488b-9f2f-e54c725603ae None None] MFA Rules not processed for user `9373333a1fd64852a138b3f5913168c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:43.514432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b55b105a-f421-488b-9f2f-e54c725603ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:43.638804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:43.666280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 853/1725] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:43.667719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d754076-1780-4f72-8142-a5e049e06c2d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:43.668023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d754076-1780-4f72-8142-a5e049e06c2d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:43.668692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d754076-1780-4f72-8142-a5e049e06c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:43.726416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6d754076-1780-4f72-8142-a5e049e06c2d None None] MFA Rules not processed for user `960b18fe180f4185b291da1d9c18a2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:43.728347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6d754076-1780-4f72-8142-a5e049e06c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:43.735444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e8b5b-f779-4554-b1ad-ea6a1fac8893 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:43.737708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e8b5b-f779-4554-b1ad-ea6a1fac8893 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:43.745578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 854/1726] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:43.774295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 873/1727] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 751 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:43.835721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d754076-1780-4f72-8142-a5e049e06c2d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:43.837313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6d754076-1780-4f72-8142-a5e049e06c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:43.899298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79d305-28d6-4a6d-829b-bdcf44993dc1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:43.899587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79d305-28d6-4a6d-829b-bdcf44993dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:43.899867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79d305-28d6-4a6d-829b-bdcf44993dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:43.937159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cc79d305-28d6-4a6d-829b-bdcf44993dc1 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:43.938426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cc79d305-28d6-4a6d-829b-bdcf44993dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:44.039179 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d754076-1780-4f72-8142-a5e049e06c2d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:44.057392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:44.058652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:44.059117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:44.059628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] PATH_INFO: `/v3/projects/7b88cdc82ee04393b8af77d1958590ce` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:44.061130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b88cdc82ee04393b8af77d1958590ce)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:44.063309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:44.122268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 874/1728] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:35:44.123437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79d305-28d6-4a6d-829b-bdcf44993dc1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:44.221717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79d305-28d6-4a6d-829b-bdcf44993dc1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:44.235088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b88cdc82ee04393b8af77d1958590ce', 'request_id': 'req-395b0c84-51ef-4dc3-b406-05bf083b2d2e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:44.257416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b88cdc82ee04393b8af77d1958590ce', 'request_id': 'req-395b0c84-51ef-4dc3-b406-05bf083b2d2e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:44.273445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b88cdc82ee04393b8af77d1958590ce', 'request_id': 'req-395b0c84-51ef-4dc3-b406-05bf083b2d2e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:44.298356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] The token cache is being invalidate because project 7b88cdc82ee04393b8af77d1958590ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:35:44.298612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:44.301364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:44.307533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 855/1729] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:35:43 2022] DELETE /identity/v3/projects/7b88cdc82ee04393b8af77d1958590ce => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:44.308540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:44.314586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:44.341392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:44.418205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:44.419181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:44.419635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:44.420059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:44.430369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:44.455367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b0c84-51ef-4dc3-b406-05bf083b2d2e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:44.463934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a6d17-2cf7-42a7-bbe5-2d3bbb37e500 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:44.465497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a6d17-2cf7-42a7-bbe5-2d3bbb37e500 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:44.465920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a6d17-2cf7-42a7-bbe5-2d3bbb37e500 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:44.466718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a6d17-2cf7-42a7-bbe5-2d3bbb37e500 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:44.471566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-554a6d17-2cf7-42a7-bbe5-2d3bbb37e500 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:44.491486 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a6d17-2cf7-42a7-bbe5-2d3bbb37e500 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:44.493499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a6d17-2cf7-42a7-bbe5-2d3bbb37e500 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:44.508371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 856/1730] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:44.510129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415544e5-37b6-4588-beb8-b7a9f4fdfba5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:44.510419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415544e5-37b6-4588-beb8-b7a9f4fdfba5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:44.510697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415544e5-37b6-4588-beb8-b7a9f4fdfba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:44.572874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-415544e5-37b6-4588-beb8-b7a9f4fdfba5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:44.572874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-415544e5-37b6-4588-beb8-b7a9f4fdfba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:44.581736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:44.650373 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:44.652830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:44.677302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 875/1731] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:44.678711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:44.680852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:44.783099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 857/1732] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:44.792065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953ee1-61db-4013-a015-55d26160e58c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:44.793360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953ee1-61db-4013-a015-55d26160e58c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:44.794673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953ee1-61db-4013-a015-55d26160e58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:44.856857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9b953ee1-61db-4013-a015-55d26160e58c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:44.861348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9b953ee1-61db-4013-a015-55d26160e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:44.885839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 858/1733] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:44.887598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b953ee1-61db-4013-a015-55d26160e58c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:44.892543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd5773-fdc1-490f-b9d8-72451c8615a5 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:44.906306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b953ee1-61db-4013-a015-55d26160e58c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:44.916770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1303b-e1c6-4abb-9c07-64be49231265 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:44.917449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716aaea6-3223-4cc3-8e9f-c5c203d903e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:44.917598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1303b-e1c6-4abb-9c07-64be49231265 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:44.917771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1303b-e1c6-4abb-9c07-64be49231265 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:44.918007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1303b-e1c6-4abb-9c07-64be49231265 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:44.918165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716aaea6-3223-4cc3-8e9f-c5c203d903e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:44.918406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716aaea6-3223-4cc3-8e9f-c5c203d903e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:44.921035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716aaea6-3223-4cc3-8e9f-c5c203d903e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:44.921504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f4e1303b-e1c6-4abb-9c07-64be49231265 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:44.921855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716aaea6-3223-4cc3-8e9f-c5c203d903e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:44.922174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716aaea6-3223-4cc3-8e9f-c5c203d903e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:44.927927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716aaea6-3223-4cc3-8e9f-c5c203d903e1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:44.964938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 859/1734] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:44.966518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716aaea6-3223-4cc3-8e9f-c5c203d903e1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:44.984909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716aaea6-3223-4cc3-8e9f-c5c203d903e1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:45.012461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e004f-8927-4c5b-9480-907ab87f00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:45.013226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e004f-8927-4c5b-9480-907ab87f00bd admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:45.013602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e004f-8927-4c5b-9480-907ab87f00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:45.014001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e004f-8927-4c5b-9480-907ab87f00bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:45.014711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e004f-8927-4c5b-9480-907ab87f00bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:45.017044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e004f-8927-4c5b-9480-907ab87f00bd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:45.073727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1303b-e1c6-4abb-9c07-64be49231265 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:45.074953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 860/1735] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:44 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:45.075467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1303b-e1c6-4abb-9c07-64be49231265 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:45.078351 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e004f-8927-4c5b-9480-907ab87f00bd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:45.109610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 876/1736] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:35:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:35:45.281329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e004f-8927-4c5b-9480-907ab87f00bd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:45.294516 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43460cda-3f27-4c41-8c7d-623d8a53588a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:45.295223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460cda-3f27-4c41-8c7d-623d8a53588a admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:45.295506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460cda-3f27-4c41-8c7d-623d8a53588a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:45.296788 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460cda-3f27-4c41-8c7d-623d8a53588a admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:45.297536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43460cda-3f27-4c41-8c7d-623d8a53588a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:45.299705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43460cda-3f27-4c41-8c7d-623d8a53588a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:45.365503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 861/1737] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:45 2022] POST /identity/v3/users => generated 283 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:45.370020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1303b-e1c6-4abb-9c07-64be49231265 service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:45.386500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1303b-e1c6-4abb-9c07-64be49231265 service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:45.408016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f77ee-18ff-43cc-8e64-bf3e8acaae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:45.411566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f77ee-18ff-43cc-8e64-bf3e8acaae52 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:45.412063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f77ee-18ff-43cc-8e64-bf3e8acaae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:45.412587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f77ee-18ff-43cc-8e64-bf3e8acaae52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:45.413537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f77ee-18ff-43cc-8e64-bf3e8acaae52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:45.416229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f77ee-18ff-43cc-8e64-bf3e8acaae52 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:45.460899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 877/1738] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:45.465740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43460cda-3f27-4c41-8c7d-623d8a53588a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:45.487096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43460cda-3f27-4c41-8c7d-623d8a53588a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:45.500011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44476314-da2e-4a59-bb72-b78d38f44184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:45.501473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44476314-da2e-4a59-bb72-b78d38f44184 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:45.501837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44476314-da2e-4a59-bb72-b78d38f44184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:45.502271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44476314-da2e-4a59-bb72-b78d38f44184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:45.503033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44476314-da2e-4a59-bb72-b78d38f44184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:45.504688 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44476314-da2e-4a59-bb72-b78d38f44184 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:45.557070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 862/1739] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:45.561683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f77ee-18ff-43cc-8e64-bf3e8acaae52 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:45.587175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f77ee-18ff-43cc-8e64-bf3e8acaae52 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:45.595707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45776e-e196-453a-a14f-5f5735c39b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:45.596322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45776e-e196-453a-a14f-5f5735c39b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:45.596601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45776e-e196-453a-a14f-5f5735c39b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:45.596910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45776e-e196-453a-a14f-5f5735c39b16 admin admin] PATH_INFO: `/v3/projects/0df24ac599de4b4f960d8450893195fd/users/bdbb9500b5a54d91bd78e7ba2d3d935a/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:45.621183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:45.661124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45776e-e196-453a-a14f-5f5735c39b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df24ac599de4b4f960d8450893195fd, user_id=bdbb9500b5a54d91bd78e7ba2d3d935a, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:45.664772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45776e-e196-453a-a14f-5f5735c39b16 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:45.713737 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 878/1740] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:45 2022] PUT /identity/v3/projects/0df24ac599de4b4f960d8450893195fd/users/bdbb9500b5a54d91bd78e7ba2d3d935a/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:45.727013 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0ca61-31e5-4d3a-825a-5714364e2b92 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:45.728287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0ca61-31e5-4d3a-825a-5714364e2b92 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:45.729032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0ca61-31e5-4d3a-825a-5714364e2b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:45.799577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-48d0ca61-31e5-4d3a-825a-5714364e2b92 None None] MFA Rules not processed for user `bdbb9500b5a54d91bd78e7ba2d3d935a`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:45.802431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-48d0ca61-31e5-4d3a-825a-5714364e2b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:45.940823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:45.961463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c3301-f013-41ac-bc7d-f40ce8090853 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:45.962357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c3301-f013-41ac-bc7d-f40ce8090853 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:45.963058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c3301-f013-41ac-bc7d-f40ce8090853 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:45.965902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 879/1741] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:35:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:35:46.034951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7c72d-9d9f-41eb-a800-3d2c677dce63 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:46.035705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7c72d-9d9f-41eb-a800-3d2c677dce63 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:46.036512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7c72d-9d9f-41eb-a800-3d2c677dce63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:46.054285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 863/1742] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:46.055662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0ca61-31e5-4d3a-825a-5714364e2b92 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:46.126838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b5d7c72d-9d9f-41eb-a800-3d2c677dce63 None None] MFA Rules not processed for user `9373333a1fd64852a138b3f5913168c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:46.134947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b5d7c72d-9d9f-41eb-a800-3d2c677dce63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:46.385054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0ca61-31e5-4d3a-825a-5714364e2b92 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:46.401439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8985f6-eb34-4812-a853-f5b4a0411d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:46.402459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8985f6-eb34-4812-a853-f5b4a0411d5d admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:46.402747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8985f6-eb34-4812-a853-f5b4a0411d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:46.403026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8985f6-eb34-4812-a853-f5b4a0411d5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:46.403746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8985f6-eb34-4812-a853-f5b4a0411d5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:46.414765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8985f6-eb34-4812-a853-f5b4a0411d5d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:46.447085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 880/1743] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:35:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:46.451535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1755699-a9da-4948-955c-8905fb9f258b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:46.452110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1755699-a9da-4948-955c-8905fb9f258b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:46.452685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1755699-a9da-4948-955c-8905fb9f258b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:46.457860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 881/1744] 10.222.0.69 () {54 vars in 912 bytes} [Fri Jun 24 10:35:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:35:46.464165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1755699-a9da-4948-955c-8905fb9f258b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:46.474094 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e1755699-a9da-4948-955c-8905fb9f258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:46.519043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1755699-a9da-4948-955c-8905fb9f258b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:46.524838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 864/1745] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:46 2022] POST /identity/v3/projects => generated 363 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:46.527440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250588d-1846-42f5-bdad-f5bf390d1322 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:46.528473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250588d-1846-42f5-bdad-f5bf390d1322 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:46.530143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:46.530882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250588d-1846-42f5-bdad-f5bf390d1322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:46.531975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:46.532465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:46.533026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] PATH_INFO: `/v3/users/51c5d934c4e641578f9fb7a72731584f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:46.539041 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c5d934c4e641578f9fb7a72731584f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:46.542755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:46.607764 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6250588d-1846-42f5-bdad-f5bf390d1322 None None] MFA Rules not processed for user `a86de22b0e324c70aad792ec6b1a7e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:46.611527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6250588d-1846-42f5-bdad-f5bf390d1322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:46.742544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c5d934c4e641578f9fb7a72731584f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:46.753711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 865/1746] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:35:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:46.754902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250588d-1846-42f5-bdad-f5bf390d1322 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:46.767622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c5d934c4e641578f9fb7a72731584f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:46.780973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c5d934c4e641578f9fb7a72731584f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:46.782217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c5d934c4e641578f9fb7a72731584f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:46.814624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c5d934c4e641578f9fb7a72731584f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:46.825884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 882/1747] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:46 2022] DELETE /identity/v3/users/51c5d934c4e641578f9fb7a72731584f => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:46.826816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:46.828890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:46.982282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:46.997680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b567ec-129a-4db9-a74b-4dded6673170 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:47.009517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250588d-1846-42f5-bdad-f5bf390d1322 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:47.013983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980d14e-7114-4a49-a0ff-b253061086e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:47.014832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980d14e-7114-4a49-a0ff-b253061086e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:47.015233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980d14e-7114-4a49-a0ff-b253061086e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:47.015624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980d14e-7114-4a49-a0ff-b253061086e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:47.016449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980d14e-7114-4a49-a0ff-b253061086e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:47.019028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980d14e-7114-4a49-a0ff-b253061086e0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:47.045550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:47.061640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 883/1748] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:46 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:47.065872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980d14e-7114-4a49-a0ff-b253061086e0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:47.092376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980d14e-7114-4a49-a0ff-b253061086e0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:47.104442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceaada0-4e26-4f34-b547-68f3211c6890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:47.105366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceaada0-4e26-4f34-b547-68f3211c6890 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:47.105693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceaada0-4e26-4f34-b547-68f3211c6890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:47.106079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceaada0-4e26-4f34-b547-68f3211c6890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:47.107699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceaada0-4e26-4f34-b547-68f3211c6890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:47.111350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceaada0-4e26-4f34-b547-68f3211c6890 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:47.140302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 884/1749] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:47.141462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceaada0-4e26-4f34-b547-68f3211c6890 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:47.167167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c0243-e83b-42ad-9de1-90eddc2b9e1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:47.168156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceaada0-4e26-4f34-b547-68f3211c6890 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:47.168564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c0243-e83b-42ad-9de1-90eddc2b9e1e service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:47.168948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c0243-e83b-42ad-9de1-90eddc2b9e1e service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:47.169334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c0243-e83b-42ad-9de1-90eddc2b9e1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:47.172050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f63c0243-e83b-42ad-9de1-90eddc2b9e1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:47.188377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e726b-0263-457c-9fa0-20630ec0ee88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:47.189793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e726b-0263-457c-9fa0-20630ec0ee88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:47.190572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e726b-0263-457c-9fa0-20630ec0ee88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:47.190572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e726b-0263-457c-9fa0-20630ec0ee88 admin admin] PATH_INFO: `/v3/projects/043b995a9fa74c479aee0b13fc8741ef/users/09fcdd07c8c34404be6e5baffd987281/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:47.198907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c0243-e83b-42ad-9de1-90eddc2b9e1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:47.200656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c0243-e83b-42ad-9de1-90eddc2b9e1e service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:47.223524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 866/1750] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:47.262702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e726b-0263-457c-9fa0-20630ec0ee88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=043b995a9fa74c479aee0b13fc8741ef, user_id=09fcdd07c8c34404be6e5baffd987281, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:47.285998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e726b-0263-457c-9fa0-20630ec0ee88 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:47.338609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 885/1751] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:47 2022] PUT /identity/v3/projects/043b995a9fa74c479aee0b13fc8741ef/users/09fcdd07c8c34404be6e5baffd987281/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:47.340913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c0243-e83b-42ad-9de1-90eddc2b9e1e service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:47.365913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c0243-e83b-42ad-9de1-90eddc2b9e1e service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:47.582488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e726b-0263-457c-9fa0-20630ec0ee88 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:47.584635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0d1e726b-0263-457c-9fa0-20630ec0ee88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:47.640914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc82f2d-15f1-4598-b3cf-5168e4e7b30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:47.642603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc82f2d-15f1-4598-b3cf-5168e4e7b30f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:47.643205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc82f2d-15f1-4598-b3cf-5168e4e7b30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:47.643716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc82f2d-15f1-4598-b3cf-5168e4e7b30f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:47.647624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc82f2d-15f1-4598-b3cf-5168e4e7b30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:47.666195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc82f2d-15f1-4598-b3cf-5168e4e7b30f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:47.717651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 867/1752] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:47.721011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc82f2d-15f1-4598-b3cf-5168e4e7b30f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:47.728482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-efc82f2d-15f1-4598-b3cf-5168e4e7b30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:47.796413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc82f2d-15f1-4598-b3cf-5168e4e7b30f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:47.817450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:47.818354 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:47.821034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:47.821391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] PATH_INFO: `/v3/projects/37f6417cf2ec4f84aa826133fcece108` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:47.823914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e726b-0263-457c-9fa0-20630ec0ee88 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:47.829378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f6417cf2ec4f84aa826133fcece108)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:47.834502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:47.835405 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d152b-8734-4db2-9e9d-bf7058da3918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:47.838420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d152b-8734-4db2-9e9d-bf7058da3918 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:47.844387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d152b-8734-4db2-9e9d-bf7058da3918 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:47.845091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d152b-8734-4db2-9e9d-bf7058da3918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:47.851635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ce7d152b-8734-4db2-9e9d-bf7058da3918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:47.887472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d152b-8734-4db2-9e9d-bf7058da3918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:47.895023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d152b-8734-4db2-9e9d-bf7058da3918 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:47.961929 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 886/1753] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:35:47.963821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d152b-8734-4db2-9e9d-bf7058da3918 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:48.012610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d152b-8734-4db2-9e9d-bf7058da3918 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:48.043719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f6417cf2ec4f84aa826133fcece108', 'request_id': 'req-1c22207b-95a4-4ab8-b999-df40854aad46'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:48.074437 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f6417cf2ec4f84aa826133fcece108', 'request_id': 'req-1c22207b-95a4-4ab8-b999-df40854aad46'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:48.099049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f6417cf2ec4f84aa826133fcece108', 'request_id': 'req-1c22207b-95a4-4ab8-b999-df40854aad46'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:48.119894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] The token cache is being invalidate because project 37f6417cf2ec4f84aa826133fcece108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:35:48.120342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:48.127296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:48.127296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 868/1754] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:35:47 2022] DELETE /identity/v3/projects/37f6417cf2ec4f84aa826133fcece108 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:48.143586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:48.150255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:48.270017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:48.336038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449d27d-13f7-47af-98f4-6df4a5328b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:48.343509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d27d-13f7-47af-98f4-6df4a5328b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:48.343808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d27d-13f7-47af-98f4-6df4a5328b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:48.348443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d27d-13f7-47af-98f4-6df4a5328b8d admin admin] PATH_INFO: `/v3/projects/043b995a9fa74c479aee0b13fc8741ef/users/09fcdd07c8c34404be6e5baffd987281/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:48.356408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449d27d-13f7-47af-98f4-6df4a5328b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=043b995a9fa74c479aee0b13fc8741ef, user_id=09fcdd07c8c34404be6e5baffd987281, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:48.369488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449d27d-13f7-47af-98f4-6df4a5328b8d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:48.432324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 887/1755] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:47 2022] PUT /identity/v3/projects/043b995a9fa74c479aee0b13fc8741ef/users/09fcdd07c8c34404be6e5baffd987281/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:48.436455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bbb86-8ffa-4a35-937a-c1281f0853cd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:48.436792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bbb86-8ffa-4a35-937a-c1281f0853cd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:48.437083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bbb86-8ffa-4a35-937a-c1281f0853cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:48.469529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22207b-95a4-4ab8-b999-df40854aad46 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:48.529028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c97bbb86-8ffa-4a35-937a-c1281f0853cd None None] MFA Rules not processed for user `09fcdd07c8c34404be6e5baffd987281`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:48.531001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c97bbb86-8ffa-4a35-937a-c1281f0853cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:48.665540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:48.727542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:48.730647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:48.732114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:48.732635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] PATH_INFO: `/v3/users/6538f5a08e354ae6aabf39d8f37d2782` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:48.737209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6538f5a08e354ae6aabf39d8f37d2782)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:48.746724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:48.863468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 888/1756] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:48 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:35:48.880204 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bbb86-8ffa-4a35-937a-c1281f0853cd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:48.884958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c97bbb86-8ffa-4a35-937a-c1281f0853cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:48.980324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6538f5a08e354ae6aabf39d8f37d2782'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:49.046813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6538f5a08e354ae6aabf39d8f37d2782'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:49.093946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6538f5a08e354ae6aabf39d8f37d2782'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:49.107393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6538f5a08e354ae6aabf39d8f37d2782'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:49.109925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6538f5a08e354ae6aabf39d8f37d2782'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:49.130991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 869/1757] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:48 2022] DELETE /identity/v3/users/6538f5a08e354ae6aabf39d8f37d2782 => generated 0 bytes in 1000 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:49.144231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bbb86-8ffa-4a35-937a-c1281f0853cd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:49.302512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:49.303212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:49.303502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:49.303787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:49.310899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:49.477196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:49.480911 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:49.526068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 889/1758] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:49.681927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:49.681927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:49.719283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ab63f-cf74-427d-ad5e-4f09c82383b2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:49.743693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ada143-c1c6-4975-8f0d-b4b4676fef39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:49.745387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ada143-c1c6-4975-8f0d-b4b4676fef39 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:49.745926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ada143-c1c6-4975-8f0d-b4b4676fef39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:49.746973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ada143-c1c6-4975-8f0d-b4b4676fef39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:49.755826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-01ada143-c1c6-4975-8f0d-b4b4676fef39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:49.961161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:50.002476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ada143-c1c6-4975-8f0d-b4b4676fef39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:50.004281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ada143-c1c6-4975-8f0d-b4b4676fef39 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:50.019720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 870/1759] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:50.645146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:50.648148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:50.910058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:51.019130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808afb5-7a85-487a-bd54-46c558b36f9b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:51.020191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808afb5-7a85-487a-bd54-46c558b36f9b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:51.021734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808afb5-7a85-487a-bd54-46c558b36f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:51.030803 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79e06d-3b8a-45ac-9c41-70dba13a59c6 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:51.041109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:51.043486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:51.043781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:51.044063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] PATH_INFO: `/v3/projects/9b9fecd90abb447396a888e466273e92` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:51.045936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9fecd90abb447396a888e466273e92)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:51.051313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:51.055953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4808afb5-7a85-487a-bd54-46c558b36f9b None None] MFA Rules not processed for user `a86de22b0e324c70aad792ec6b1a7e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:51.059466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4808afb5-7a85-487a-bd54-46c558b36f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:51.211847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9fecd90abb447396a888e466273e92', 'request_id': 'req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:51.230363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9fecd90abb447396a888e466273e92', 'request_id': 'req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:51.237580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 871/1760] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Jun 24 10:35:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:51.242158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9fecd90abb447396a888e466273e92', 'request_id': 'req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:51.252870 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808afb5-7a85-487a-bd54-46c558b36f9b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:51.254866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4808afb5-7a85-487a-bd54-46c558b36f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:51.258767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] The token cache is being invalidate because project 9b9fecd90abb447396a888e466273e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:35:51.259068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:51.260095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ae4fe1f6-1d71-4547-a15a-f2e1d5506126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:51.278559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808afb5-7a85-487a-bd54-46c558b36f9b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:51.297842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 890/1761] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:35:50 2022] DELETE /identity/v3/projects/9b9fecd90abb447396a888e466273e92 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:51.303266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a48a31-df25-425d-bf76-3a1f8eb1de10 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:51.303614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a48a31-df25-425d-bf76-3a1f8eb1de10 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:51.307085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a48a31-df25-425d-bf76-3a1f8eb1de10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:51.405974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-59a48a31-df25-425d-bf76-3a1f8eb1de10 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:51.410536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-59a48a31-df25-425d-bf76-3a1f8eb1de10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:51.493824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69f85d-47c2-4ab9-a800-d8d03e638501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:51.503543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69f85d-47c2-4ab9-a800-d8d03e638501 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:51.509903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69f85d-47c2-4ab9-a800-d8d03e638501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:51.511118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69f85d-47c2-4ab9-a800-d8d03e638501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:51.516166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7f69f85d-47c2-4ab9-a800-d8d03e638501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:51.555973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69f85d-47c2-4ab9-a800-d8d03e638501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:51.557986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69f85d-47c2-4ab9-a800-d8d03e638501 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:51.591977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 872/1762] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:35:51.618428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e0b61-add9-4054-a1ca-88a7b8175fc2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:51.618428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e0b61-add9-4054-a1ca-88a7b8175fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:51.618428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e0b61-add9-4054-a1ca-88a7b8175fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:51.645116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 891/1763] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:51.646706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3861de8-8521-4b1f-9f99-0181c09a219f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:51.647254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3861de8-8521-4b1f-9f99-0181c09a219f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:51.647254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3861de8-8521-4b1f-9f99-0181c09a219f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:51.717295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f3861de8-8521-4b1f-9f99-0181c09a219f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:51.718168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f3861de8-8521-4b1f-9f99-0181c09a219f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:51.731683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-059e0b61-add9-4054-a1ca-88a7b8175fc2 None None] MFA Rules not processed for user `960b18fe180f4185b291da1d9c18a2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:51.732876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-059e0b61-add9-4054-a1ca-88a7b8175fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:51.778155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 892/1764] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:51.779136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3861de8-8521-4b1f-9f99-0181c09a219f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:51.828582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3861de8-8521-4b1f-9f99-0181c09a219f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:51.869814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83316b-6e88-459d-8f59-0a5b9c891920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:51.870527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83316b-6e88-459d-8f59-0a5b9c891920 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:51.870812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83316b-6e88-459d-8f59-0a5b9c891920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:51.871186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83316b-6e88-459d-8f59-0a5b9c891920 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:51.872169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83316b-6e88-459d-8f59-0a5b9c891920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:35:51.872671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83316b-6e88-459d-8f59-0a5b9c891920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:51.903742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83316b-6e88-459d-8f59-0a5b9c891920 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:51.987885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 893/1765] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:35:51.989542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138714a4-25cf-46c6-8456-45d985ee35b9 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:51.989839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138714a4-25cf-46c6-8456-45d985ee35b9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:51.990249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138714a4-25cf-46c6-8456-45d985ee35b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:52.086942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-138714a4-25cf-46c6-8456-45d985ee35b9 None None] MFA Rules not processed for user `3e4ed08c1c81453abaa6c1dd826f0258`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:52.098548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-138714a4-25cf-46c6-8456-45d985ee35b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:52.159708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 873/1766] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 567 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:52.160709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e0b61-add9-4054-a1ca-88a7b8175fc2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:52.224304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e0b61-add9-4054-a1ca-88a7b8175fc2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:52.249568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae395b-6b97-4aa4-a3d0-d38df15cb54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:52.250741 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae395b-6b97-4aa4-a3d0-d38df15cb54c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:52.251199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae395b-6b97-4aa4-a3d0-d38df15cb54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:52.251688 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae395b-6b97-4aa4-a3d0-d38df15cb54c admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:52.253171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae395b-6b97-4aa4-a3d0-d38df15cb54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:52.255271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae395b-6b97-4aa4-a3d0-d38df15cb54c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:52.340125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 874/1767] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:52 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:52.344562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae395b-6b97-4aa4-a3d0-d38df15cb54c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:52.351812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:52.382719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 894/1768] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:52.445296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:52.628037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae395b-6b97-4aa4-a3d0-d38df15cb54c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:52.653509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968407-232e-410c-b675-45153f9bdd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:52.664508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968407-232e-410c-b675-45153f9bdd91 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:52.664508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968407-232e-410c-b675-45153f9bdd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:52.664508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968407-232e-410c-b675-45153f9bdd91 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:52.664508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d968407-232e-410c-b675-45153f9bdd91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:52.678196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d968407-232e-410c-b675-45153f9bdd91 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:52.737794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 875/1769] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:52 2022] POST /identity/v3/users => generated 293 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:52.741756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138714a4-25cf-46c6-8456-45d985ee35b9 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:52.763960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138714a4-25cf-46c6-8456-45d985ee35b9 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:52.780012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfde8e-f798-4771-ad35-c7111b820366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:52.780922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfde8e-f798-4771-ad35-c7111b820366 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:52.783236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfde8e-f798-4771-ad35-c7111b820366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:52.784039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfde8e-f798-4771-ad35-c7111b820366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:52.786116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfde8e-f798-4771-ad35-c7111b820366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:52.790520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfde8e-f798-4771-ad35-c7111b820366 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:52.833721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 895/1770] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:52.840308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968407-232e-410c-b675-45153f9bdd91 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:52.869520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968407-232e-410c-b675-45153f9bdd91 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:52.882792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c2906-afb2-4fdf-a4e4-6d42f3456680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:52.882792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c2906-afb2-4fdf-a4e4-6d42f3456680 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:52.882792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c2906-afb2-4fdf-a4e4-6d42f3456680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:52.882792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c2906-afb2-4fdf-a4e4-6d42f3456680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:52.883821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c2906-afb2-4fdf-a4e4-6d42f3456680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:52.883909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c2906-afb2-4fdf-a4e4-6d42f3456680 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:52.950408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 876/1771] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:52.954534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfde8e-f798-4771-ad35-c7111b820366 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:52.984040 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfde8e-f798-4771-ad35-c7111b820366 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:53.023049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0265e-0411-426e-9636-0e07679e2917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:53.026554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0265e-0411-426e-9636-0e07679e2917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:53.031283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0265e-0411-426e-9636-0e07679e2917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:53.035069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0265e-0411-426e-9636-0e07679e2917 admin admin] PATH_INFO: `/v3/projects/810b447768eb4c5f84b81078a79766bb/users/aeb122f4467347dc97bc2694e318b81d/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:53.087707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0265e-0411-426e-9636-0e07679e2917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=810b447768eb4c5f84b81078a79766bb, user_id=aeb122f4467347dc97bc2694e318b81d, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:53.095867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0265e-0411-426e-9636-0e07679e2917 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:53.146116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 896/1772] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:52 2022] PUT /identity/v3/projects/810b447768eb4c5f84b81078a79766bb/users/aeb122f4467347dc97bc2694e318b81d/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:53.147814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ac136-9e62-4a04-ac10-17412e5d6f8f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:53.148344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ac136-9e62-4a04-ac10-17412e5d6f8f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:53.148861 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ac136-9e62-4a04-ac10-17412e5d6f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:53.241617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d83ac136-9e62-4a04-ac10-17412e5d6f8f None None] MFA Rules not processed for user `aeb122f4467347dc97bc2694e318b81d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:53.243049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d83ac136-9e62-4a04-ac10-17412e5d6f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:53.446551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 877/1773] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:35:53.447506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0265e-0411-426e-9636-0e07679e2917 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:53.657450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0265e-0411-426e-9636-0e07679e2917 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:53.664850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495c0f3-bf5c-421a-9dde-6671678c43f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:53.665890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495c0f3-bf5c-421a-9dde-6671678c43f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:53.666375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495c0f3-bf5c-421a-9dde-6671678c43f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:53.667069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495c0f3-bf5c-421a-9dde-6671678c43f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:53.668271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495c0f3-bf5c-421a-9dde-6671678c43f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:53.670569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495c0f3-bf5c-421a-9dde-6671678c43f6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:53.722695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 897/1774] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:53 2022] POST /identity/v3/projects => generated 383 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:35:53.726659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ac136-9e62-4a04-ac10-17412e5d6f8f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:53.746078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ac136-9e62-4a04-ac10-17412e5d6f8f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:53.902559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1ddc9-fa51-4f46-b9c1-7cea411ce3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:53.903463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1ddc9-fa51-4f46-b9c1-7cea411ce3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:53.905233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1ddc9-fa51-4f46-b9c1-7cea411ce3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:53.905589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1ddc9-fa51-4f46-b9c1-7cea411ce3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:53.907233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1ddc9-fa51-4f46-b9c1-7cea411ce3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:53.914170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1ddc9-fa51-4f46-b9c1-7cea411ce3c2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:53.962736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 878/1775] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:53 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:53.966931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495c0f3-bf5c-421a-9dde-6671678c43f6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:53.982150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495c0f3-bf5c-421a-9dde-6671678c43f6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:53.989454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf9d86-8358-494b-8fdd-f1627823b4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:53.990656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf9d86-8358-494b-8fdd-f1627823b4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:53.991144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf9d86-8358-494b-8fdd-f1627823b4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:53.991631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf9d86-8358-494b-8fdd-f1627823b4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:53.992902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf9d86-8358-494b-8fdd-f1627823b4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:53.995529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf9d86-8358-494b-8fdd-f1627823b4a6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:54.016088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 898/1776] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:54.020511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1ddc9-fa51-4f46-b9c1-7cea411ce3c2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:54.078170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1ddc9-fa51-4f46-b9c1-7cea411ce3c2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:54.091623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11a597-4f3f-47cd-924c-2f7973d872c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:54.094200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11a597-4f3f-47cd-924c-2f7973d872c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:54.097927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11a597-4f3f-47cd-924c-2f7973d872c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:54.098935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11a597-4f3f-47cd-924c-2f7973d872c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:54.101357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11a597-4f3f-47cd-924c-2f7973d872c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:54.105745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11a597-4f3f-47cd-924c-2f7973d872c9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:54.153380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 879/1777] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:35:54.157888 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf9d86-8358-494b-8fdd-f1627823b4a6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:54.175484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf9d86-8358-494b-8fdd-f1627823b4a6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:54.185189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715dff5-017e-4842-a1b0-6a6e970efd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:54.185994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715dff5-017e-4842-a1b0-6a6e970efd5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:54.186279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715dff5-017e-4842-a1b0-6a6e970efd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:54.186557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715dff5-017e-4842-a1b0-6a6e970efd5f admin admin] PATH_INFO: `/v3/projects/e3c9429879e3458aa5e7511e60e541c7/users/6cef4fae28f54171b9839d65393c8f32/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:54.233897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715dff5-017e-4842-a1b0-6a6e970efd5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c9429879e3458aa5e7511e60e541c7, user_id=6cef4fae28f54171b9839d65393c8f32, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:54.238523 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715dff5-017e-4842-a1b0-6a6e970efd5f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:54.275437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 899/1778] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:54 2022] PUT /identity/v3/projects/e3c9429879e3458aa5e7511e60e541c7/users/6cef4fae28f54171b9839d65393c8f32/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:54.285910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac2e92-dc9c-497b-87ae-46e9c8997439 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:54.286384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac2e92-dc9c-497b-87ae-46e9c8997439 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:54.286581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac2e92-dc9c-497b-87ae-46e9c8997439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:54.324839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:54.343874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a9ac2e92-dc9c-497b-87ae-46e9c8997439 None None] MFA Rules not processed for user `6cef4fae28f54171b9839d65393c8f32`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:54.345434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a9ac2e92-dc9c-497b-87ae-46e9c8997439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:54.379441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715dff5-017e-4842-a1b0-6a6e970efd5f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:54.381425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2715dff5-017e-4842-a1b0-6a6e970efd5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:54.531055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 880/1779] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:54.535631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac2e92-dc9c-497b-87ae-46e9c8997439 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:54.541168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715dff5-017e-4842-a1b0-6a6e970efd5f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:54.555742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:54.556965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac2e92-dc9c-497b-87ae-46e9c8997439 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:54.557623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:54.558271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:54.558771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] PATH_INFO: `/v3/users/bdbb9500b5a54d91bd78e7ba2d3d935a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:54.561881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdbb9500b5a54d91bd78e7ba2d3d935a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:54.564706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fcae2-f6f7-4323-9000-0f6b6ceaaead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:54.565656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fcae2-f6f7-4323-9000-0f6b6ceaaead admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:54.566020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fcae2-f6f7-4323-9000-0f6b6ceaaead admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:54.566370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fcae2-f6f7-4323-9000-0f6b6ceaaead admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:54.567602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fcae2-f6f7-4323-9000-0f6b6ceaaead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:54.569016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:54.572214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fcae2-f6f7-4323-9000-0f6b6ceaaead admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:54.583320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:54.644472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 881/1780] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:35:54 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:54.648475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fcae2-f6f7-4323-9000-0f6b6ceaaead admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:54.659254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdbb9500b5a54d91bd78e7ba2d3d935a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:54.684460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdbb9500b5a54d91bd78e7ba2d3d935a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:54.700150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdbb9500b5a54d91bd78e7ba2d3d935a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:54.701712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdbb9500b5a54d91bd78e7ba2d3d935a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:54.721284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f8ef842f-6a48-406d-8285-2c2cc52228db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdbb9500b5a54d91bd78e7ba2d3d935a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:54.741976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 900/1781] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:54 2022] DELETE /identity/v3/users/bdbb9500b5a54d91bd78e7ba2d3d935a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:35:54.744776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1b9ea-3c4f-407f-a6f9-6ff6c31f2eb0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:54.744776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1b9ea-3c4f-407f-a6f9-6ff6c31f2eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:54.744971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1b9ea-3c4f-407f-a6f9-6ff6c31f2eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:54.804631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9cc1b9ea-3c4f-407f-a6f9-6ff6c31f2eb0 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:54.807185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9cc1b9ea-3c4f-407f-a6f9-6ff6c31f2eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:54.854703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fcae2-f6f7-4323-9000-0f6b6ceaaead admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:55.048120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5ffb4-76eb-4262-9cb4-151287f5bfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:55.051781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5ffb4-76eb-4262-9cb4-151287f5bfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:55.052080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5ffb4-76eb-4262-9cb4-151287f5bfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:55.052368 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5ffb4-76eb-4262-9cb4-151287f5bfd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:55.053147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5ffb4-76eb-4262-9cb4-151287f5bfd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:55.074320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5ffb4-76eb-4262-9cb4-151287f5bfd1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:55.123914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 901/1782] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:55.147537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 882/1783] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:35:54 2022] POST /identity/v3/users => generated 300 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:35:55.151048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1b9ea-3c4f-407f-a6f9-6ff6c31f2eb0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:55.152260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9cc1b9ea-3c4f-407f-a6f9-6ff6c31f2eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:55.180313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1b9ea-3c4f-407f-a6f9-6ff6c31f2eb0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:55.184032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5ffb4-76eb-4262-9cb4-151287f5bfd1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:55.185846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6fa5ffb4-76eb-4262-9cb4-151287f5bfd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:55.193335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006603c2-f714-442b-8ea6-1f20c25ddcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:55.194037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006603c2-f714-442b-8ea6-1f20c25ddcbf admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:55.194321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006603c2-f714-442b-8ea6-1f20c25ddcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:55.194610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006603c2-f714-442b-8ea6-1f20c25ddcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:55.195350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006603c2-f714-442b-8ea6-1f20c25ddcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:55.198980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006603c2-f714-442b-8ea6-1f20c25ddcbf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:55.224004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 902/1784] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:55.224986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006603c2-f714-442b-8ea6-1f20c25ddcbf admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:55.244583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:55.250295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006603c2-f714-442b-8ea6-1f20c25ddcbf admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:55.256311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb999525-7532-446f-b9c2-ded67d2b5fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:55.257384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb999525-7532-446f-b9c2-ded67d2b5fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:55.257656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb999525-7532-446f-b9c2-ded67d2b5fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:55.257926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb999525-7532-446f-b9c2-ded67d2b5fbb admin admin] PATH_INFO: `/v3/projects/e120b72afd2a40f9b61154b62bcc00f0/users/9fb8b53c0bf74db081f3a1ecf205f7a6/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:55.279797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb999525-7532-446f-b9c2-ded67d2b5fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e120b72afd2a40f9b61154b62bcc00f0, user_id=9fb8b53c0bf74db081f3a1ecf205f7a6, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:55.283487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb999525-7532-446f-b9c2-ded67d2b5fbb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:55.316660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 903/1785] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:55 2022] PUT /identity/v3/projects/e120b72afd2a40f9b61154b62bcc00f0/users/9fb8b53c0bf74db081f3a1ecf205f7a6/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:55.317674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb999525-7532-446f-b9c2-ded67d2b5fbb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:55.428877 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5ffb4-76eb-4262-9cb4-151287f5bfd1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:55.544219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:55.597723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb999525-7532-446f-b9c2-ded67d2b5fbb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:55.610606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f825364-ef86-4f6c-9b50-757c5e87bbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:55.612002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f825364-ef86-4f6c-9b50-757c5e87bbfa admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:55.612545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f825364-ef86-4f6c-9b50-757c5e87bbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:55.613238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f825364-ef86-4f6c-9b50-757c5e87bbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:55.615317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f825364-ef86-4f6c-9b50-757c5e87bbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:55.617712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f825364-ef86-4f6c-9b50-757c5e87bbfa admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:55.617901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7ad4d-6182-4875-9b31-7674a76461d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:55.618720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ad4d-6182-4875-9b31-7674a76461d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:55.619352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ad4d-6182-4875-9b31-7674a76461d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:55.623473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ad4d-6182-4875-9b31-7674a76461d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:55.654469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7ad4d-6182-4875-9b31-7674a76461d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:55.655967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7ad4d-6182-4875-9b31-7674a76461d1 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:55.661544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 904/1786] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:35:55.667400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f825364-ef86-4f6c-9b50-757c5e87bbfa admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:55.682600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 883/1787] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:55.685895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f825364-ef86-4f6c-9b50-757c5e87bbfa admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:55.706337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea66bd-f49e-4c07-ad81-ff62939dce93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:55.733849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea66bd-f49e-4c07-ad81-ff62939dce93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:55.734871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea66bd-f49e-4c07-ad81-ff62939dce93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:55.737568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea66bd-f49e-4c07-ad81-ff62939dce93 admin admin] PATH_INFO: `/v3/projects/e120b72afd2a40f9b61154b62bcc00f0/users/9fb8b53c0bf74db081f3a1ecf205f7a6/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:55.765407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea66bd-f49e-4c07-ad81-ff62939dce93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e120b72afd2a40f9b61154b62bcc00f0, user_id=9fb8b53c0bf74db081f3a1ecf205f7a6, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:55.789834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea66bd-f49e-4c07-ad81-ff62939dce93 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:55.900978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 905/1788] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:35:55 2022] PUT /identity/v3/projects/e120b72afd2a40f9b61154b62bcc00f0/users/9fb8b53c0bf74db081f3a1ecf205f7a6/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:35:55.904335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bff24-68f0-4bed-9cf1-b5eaaafff428 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:55.904628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bff24-68f0-4bed-9cf1-b5eaaafff428 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:55.905056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bff24-68f0-4bed-9cf1-b5eaaafff428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:56.081433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-183bff24-68f0-4bed-9cf1-b5eaaafff428 None None] MFA Rules not processed for user `9fb8b53c0bf74db081f3a1ecf205f7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:56.082776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-183bff24-68f0-4bed-9cf1-b5eaaafff428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:56.389179 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 884/1789] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:35:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:56.407827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea66bd-f49e-4c07-ad81-ff62939dce93 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:56.410726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-16ea66bd-f49e-4c07-ad81-ff62939dce93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:56.632563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea66bd-f49e-4c07-ad81-ff62939dce93 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:56.639291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b95568-6c29-42cb-a1fe-747b64750c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:56.640033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b95568-6c29-42cb-a1fe-747b64750c8c service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:56.641225 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b95568-6c29-42cb-a1fe-747b64750c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:56.641610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b95568-6c29-42cb-a1fe-747b64750c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:56.719695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b95568-6c29-42cb-a1fe-747b64750c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:56.726057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b95568-6c29-42cb-a1fe-747b64750c8c service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:56.746781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 906/1790] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:58.330721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bff24-68f0-4bed-9cf1-b5eaaafff428 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:58.349105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bff24-68f0-4bed-9cf1-b5eaaafff428 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:58.353233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff83ab-1da4-426b-b92b-ea541f0e8d80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:58.354365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff83ab-1da4-426b-b92b-ea541f0e8d80 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:58.354795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff83ab-1da4-426b-b92b-ea541f0e8d80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:58.355216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff83ab-1da4-426b-b92b-ea541f0e8d80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:58.366025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-69ff83ab-1da4-426b-b92b-ea541f0e8d80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:58.502397 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff83ab-1da4-426b-b92b-ea541f0e8d80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:58.506996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff83ab-1da4-426b-b92b-ea541f0e8d80 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:58.519813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:58.535681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 885/1791] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:58.965371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc7e60-8b66-43f8-8802-efdaa5d5835b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:58.965891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc7e60-8b66-43f8-8802-efdaa5d5835b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:58.968795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc7e60-8b66-43f8-8802-efdaa5d5835b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:58.971095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 907/1792] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:35:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:35:59.210255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43f699-fc9e-4e37-be7c-03834d464583 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:59.210696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43f699-fc9e-4e37-be7c-03834d464583 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:59.211011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43f699-fc9e-4e37-be7c-03834d464583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:59.252860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3c43f699-fc9e-4e37-be7c-03834d464583 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:35:59.254628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3c43f699-fc9e-4e37-be7c-03834d464583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:59.292914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc7e60-8b66-43f8-8802-efdaa5d5835b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:59.295161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-33bc7e60-8b66-43f8-8802-efdaa5d5835b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:59.474006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc7e60-8b66-43f8-8802-efdaa5d5835b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:59.482526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:59.484225 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:59.485095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:59.485484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] PATH_INFO: `/v3/projects/0df24ac599de4b4f960d8450893195fd` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:59.487802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df24ac599de4b4f960d8450893195fd)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:59.491950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:59.510730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 886/1793] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:35:59.593244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df24ac599de4b4f960d8450893195fd', 'request_id': 'req-8a702e9a-cb5e-474e-8b2a-926e45719add'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.616126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df24ac599de4b4f960d8450893195fd', 'request_id': 'req-8a702e9a-cb5e-474e-8b2a-926e45719add'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.625063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df24ac599de4b4f960d8450893195fd', 'request_id': 'req-8a702e9a-cb5e-474e-8b2a-926e45719add'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.644332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43f699-fc9e-4e37-be7c-03834d464583 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:59.646695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3c43f699-fc9e-4e37-be7c-03834d464583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:59.651865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] The token cache is being invalidate because project 0df24ac599de4b4f960d8450893195fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:35:59.652221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.653494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.658376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 908/1794] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:35:59 2022] DELETE /identity/v3/projects/0df24ac599de4b4f960d8450893195fd => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:35:59.660285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:35:59.664567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:59.664904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43f699-fc9e-4e37-be7c-03834d464583 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:59.684322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:35:59.764329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a702e9a-cb5e-474e-8b2a-926e45719add admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:35:59.774086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:59.775642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:59.775915 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:59.777530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] PATH_INFO: `/v3/users/09fcdd07c8c34404be6e5baffd987281` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:59.780336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=09fcdd07c8c34404be6e5baffd987281)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:59.782323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:59.792969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f4f19-d955-4c67-9d94-3450c9910db7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:35:59.793645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f4f19-d955-4c67-9d94-3450c9910db7 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:35:59.793928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f4f19-d955-4c67-9d94-3450c9910db7 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:35:59.794199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f4f19-d955-4c67-9d94-3450c9910db7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:35:59.796023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3b0f4f19-d955-4c67-9d94-3450c9910db7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:35:59.808189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f4f19-d955-4c67-9d94-3450c9910db7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:35:59.810125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f4f19-d955-4c67-9d94-3450c9910db7 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:35:59.823027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 887/1795] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:35:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:35:59.863295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09fcdd07c8c34404be6e5baffd987281'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.887883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09fcdd07c8c34404be6e5baffd987281'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.915974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09fcdd07c8c34404be6e5baffd987281'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.916673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09fcdd07c8c34404be6e5baffd987281'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.930297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5eb1a27f-e4dd-4c4b-ae31-4da04aef05bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09fcdd07c8c34404be6e5baffd987281'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:35:59.957118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 909/1796] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:35:59 2022] DELETE /identity/v3/users/09fcdd07c8c34404be6e5baffd987281 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:01.644890 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f4f19-d955-4c67-9d94-3450c9910db7 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:01.656450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f4f19-d955-4c67-9d94-3450c9910db7 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:01.743646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:01.744454 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:01.744891 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:01.745400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] PATH_INFO: `/v3/projects/043b995a9fa74c479aee0b13fc8741ef` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:01.748774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] RBAC: Authorizing `identity:delete_project(project_id=043b995a9fa74c479aee0b13fc8741ef)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:01.751985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:01.783602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:01.863700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '043b995a9fa74c479aee0b13fc8741ef', 'request_id': 'req-3333cd34-a800-4d1b-810e-a13131a6baff'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:01.873887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '043b995a9fa74c479aee0b13fc8741ef', 'request_id': 'req-3333cd34-a800-4d1b-810e-a13131a6baff'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:01.898257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '043b995a9fa74c479aee0b13fc8741ef', 'request_id': 'req-3333cd34-a800-4d1b-810e-a13131a6baff'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:01.920788 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] The token cache is being invalidate because project 043b995a9fa74c479aee0b13fc8741ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:01.921092 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:01.921850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3333cd34-a800-4d1b-810e-a13131a6baff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:01.928100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 888/1797] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:01 2022] DELETE /identity/v3/projects/043b995a9fa74c479aee0b13fc8741ef => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:01.929263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb8750-0952-4ff1-be2d-74b1e926b076 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:01.931048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb8750-0952-4ff1-be2d-74b1e926b076 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:01.931836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb8750-0952-4ff1-be2d-74b1e926b076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:01.980006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fceb8750-0952-4ff1-be2d-74b1e926b076 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:01.982764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fceb8750-0952-4ff1-be2d-74b1e926b076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:02.129017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 910/1798] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:02.148157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56af1c-fbbb-4694-92a7-8a61bc81fd4c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.148450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56af1c-fbbb-4694-92a7-8a61bc81fd4c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.148929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56af1c-fbbb-4694-92a7-8a61bc81fd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.175580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-db56af1c-fbbb-4694-92a7-8a61bc81fd4c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:02.177238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-db56af1c-fbbb-4694-92a7-8a61bc81fd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:02.193927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 889/1799] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:02.198760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb8750-0952-4ff1-be2d-74b1e926b076 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:02.223248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb8750-0952-4ff1-be2d-74b1e926b076 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:02.235890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eda7d1-922d-4996-b988-e571f32cdb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:02.239851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eda7d1-922d-4996-b988-e571f32cdb0b admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.240237 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eda7d1-922d-4996-b988-e571f32cdb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.240618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eda7d1-922d-4996-b988-e571f32cdb0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.241532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eda7d1-922d-4996-b988-e571f32cdb0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:02.241946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eda7d1-922d-4996-b988-e571f32cdb0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:02.246538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eda7d1-922d-4996-b988-e571f32cdb0b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:02.274617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 911/1800] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:02.276462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56af1c-fbbb-4694-92a7-8a61bc81fd4c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:02.291164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56af1c-fbbb-4694-92a7-8a61bc81fd4c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:02.297198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b8fc1-57ca-4172-bdbb-19d1ba161656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:02.298415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b8fc1-57ca-4172-bdbb-19d1ba161656 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.299085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b8fc1-57ca-4172-bdbb-19d1ba161656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.299693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b8fc1-57ca-4172-bdbb-19d1ba161656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.300670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b8fc1-57ca-4172-bdbb-19d1ba161656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:02.303171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b8fc1-57ca-4172-bdbb-19d1ba161656 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:02.368900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 890/1801] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:02 2022] POST /identity/v3/projects => generated 369 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:02.369801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eda7d1-922d-4996-b988-e571f32cdb0b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:02.383750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eda7d1-922d-4996-b988-e571f32cdb0b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:02.416022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:02.510609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a075dad-691d-47ae-8db4-27957a8c4623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:02.511370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a075dad-691d-47ae-8db4-27957a8c4623 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.513549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a075dad-691d-47ae-8db4-27957a8c4623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.513859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a075dad-691d-47ae-8db4-27957a8c4623 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.514593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a075dad-691d-47ae-8db4-27957a8c4623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:02.528761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a075dad-691d-47ae-8db4-27957a8c4623 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:02.564960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 912/1802] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:02 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:02.569755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b8fc1-57ca-4172-bdbb-19d1ba161656 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:02.587433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b8fc1-57ca-4172-bdbb-19d1ba161656 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:02.601122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af2a32-102a-48c5-a12e-335028acc342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:02.609711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af2a32-102a-48c5-a12e-335028acc342 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.610081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af2a32-102a-48c5-a12e-335028acc342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.610293 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af2a32-102a-48c5-a12e-335028acc342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.611311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af2a32-102a-48c5-a12e-335028acc342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:02.615742 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af2a32-102a-48c5-a12e-335028acc342 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:02.646584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 891/1803] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:02.651501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a075dad-691d-47ae-8db4-27957a8c4623 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:02.671682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a075dad-691d-47ae-8db4-27957a8c4623 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:02.686010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55bee-c529-4fbd-9a4b-c39a0c8d2620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:02.687388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55bee-c529-4fbd-9a4b-c39a0c8d2620 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.688303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55bee-c529-4fbd-9a4b-c39a0c8d2620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.688590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55bee-c529-4fbd-9a4b-c39a0c8d2620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.689746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe55bee-c529-4fbd-9a4b-c39a0c8d2620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:02.693902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe55bee-c529-4fbd-9a4b-c39a0c8d2620 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:02.715771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 913/1804] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:02.720712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af2a32-102a-48c5-a12e-335028acc342 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:02.738508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af2a32-102a-48c5-a12e-335028acc342 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:02.752077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e9950-0a5b-4135-ab37-8a80d3f845e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:02.753779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e9950-0a5b-4135-ab37-8a80d3f845e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.754332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e9950-0a5b-4135-ab37-8a80d3f845e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.754779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e9950-0a5b-4135-ab37-8a80d3f845e9 admin admin] PATH_INFO: `/v3/projects/c9525c975f4e4f0d8327fd35a9d40601/users/6dca49ea23144f739f07fc0148ee92a8/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.799998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e9950-0a5b-4135-ab37-8a80d3f845e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9525c975f4e4f0d8327fd35a9d40601, user_id=6dca49ea23144f739f07fc0148ee92a8, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:02.810038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e9950-0a5b-4135-ab37-8a80d3f845e9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:02.859564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf0239-c152-4737-835c-0544a79b69d6 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.859891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf0239-c152-4737-835c-0544a79b69d6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.860611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf0239-c152-4737-835c-0544a79b69d6 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.863022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 914/1806] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:36:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:36:02.873669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 892/1806] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:02 2022] PUT /identity/v3/projects/c9525c975f4e4f0d8327fd35a9d40601/users/6dca49ea23144f739f07fc0148ee92a8/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:02.875370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b56f95-6118-4f9e-8cb4-e4f93b1d822c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.875667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b56f95-6118-4f9e-8cb4-e4f93b1d822c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.875950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b56f95-6118-4f9e-8cb4-e4f93b1d822c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.876186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba5604-6856-4f9d-942e-e80bdb5f8889 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:02.876706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba5604-6856-4f9d-942e-e80bdb5f8889 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:02.882129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba5604-6856-4f9d-942e-e80bdb5f8889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:02.942056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-84ba5604-6856-4f9d-942e-e80bdb5f8889 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:02.942686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-99b56f95-6118-4f9e-8cb4-e4f93b1d822c None None] MFA Rules not processed for user `6dca49ea23144f739f07fc0148ee92a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:02.943487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-99b56f95-6118-4f9e-8cb4-e4f93b1d822c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:02.947443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-84ba5604-6856-4f9d-942e-e80bdb5f8889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:03.087925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 915/1807] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:36:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:03.127376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 893/1808] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:03.131984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba5604-6856-4f9d-942e-e80bdb5f8889 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:03.240066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba5604-6856-4f9d-942e-e80bdb5f8889 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:03.245890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadaeb6-8942-46aa-a223-42f1fa8f3890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:03.247427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadaeb6-8942-46aa-a223-42f1fa8f3890 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:03.247782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadaeb6-8942-46aa-a223-42f1fa8f3890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:03.248461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadaeb6-8942-46aa-a223-42f1fa8f3890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:03.249291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadaeb6-8942-46aa-a223-42f1fa8f3890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:03.251673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadaeb6-8942-46aa-a223-42f1fa8f3890 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:03.314858 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 916/1809] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:03 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:36:03.320307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b56f95-6118-4f9e-8cb4-e4f93b1d822c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:03.343727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b56f95-6118-4f9e-8cb4-e4f93b1d822c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:03.346413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadaeb6-8942-46aa-a223-42f1fa8f3890 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:03.350254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ceadaeb6-8942-46aa-a223-42f1fa8f3890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:03.390385 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:03.444142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:03.525251 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50784e6f-1f42-4332-8317-3cd655478853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:03.528005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50784e6f-1f42-4332-8317-3cd655478853 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:03.528474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50784e6f-1f42-4332-8317-3cd655478853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:03.528867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50784e6f-1f42-4332-8317-3cd655478853 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:03.531571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50784e6f-1f42-4332-8317-3cd655478853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:03.535510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50784e6f-1f42-4332-8317-3cd655478853 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:03.554084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadaeb6-8942-46aa-a223-42f1fa8f3890 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:03.564630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f85fd4-3694-444e-9f2c-07a260c09bf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:03.565483 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f85fd4-3694-444e-9f2c-07a260c09bf4 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:03.574478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f85fd4-3694-444e-9f2c-07a260c09bf4 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:03.574478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f85fd4-3694-444e-9f2c-07a260c09bf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:03.581262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 894/1810] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:03 2022] POST /identity/v3/users => generated 291 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:03.584984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50784e6f-1f42-4332-8317-3cd655478853 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:03.588542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f85fd4-3694-444e-9f2c-07a260c09bf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:03.591222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f85fd4-3694-444e-9f2c-07a260c09bf4 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:03.604075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50784e6f-1f42-4332-8317-3cd655478853 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:03.608209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 917/1811] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:36:03.610560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364cb8a-d2f1-45f5-b375-ec6026ba3c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:03.612072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364cb8a-d2f1-45f5-b375-ec6026ba3c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:03.612423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364cb8a-d2f1-45f5-b375-ec6026ba3c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:03.612787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364cb8a-d2f1-45f5-b375-ec6026ba3c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:03.613736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364cb8a-d2f1-45f5-b375-ec6026ba3c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:03.615563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364cb8a-d2f1-45f5-b375-ec6026ba3c55 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:03.668037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 895/1812] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:03.672072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f85fd4-3694-444e-9f2c-07a260c09bf4 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:03.687867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f85fd4-3694-444e-9f2c-07a260c09bf4 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:03.694431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e092c3e-cbc5-48e6-b023-72f0ac9c29b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:03.695343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e092c3e-cbc5-48e6-b023-72f0ac9c29b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:03.695627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e092c3e-cbc5-48e6-b023-72f0ac9c29b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:03.695905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e092c3e-cbc5-48e6-b023-72f0ac9c29b0 admin admin] PATH_INFO: `/v3/projects/f7a54d85c0a5472d9c96613856e7c586/users/356d5836e1b347e0a7ffcd981b8347be/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:03.723754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e092c3e-cbc5-48e6-b023-72f0ac9c29b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a54d85c0a5472d9c96613856e7c586, user_id=356d5836e1b347e0a7ffcd981b8347be, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:03.727660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e092c3e-cbc5-48e6-b023-72f0ac9c29b0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:03.747457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 918/1813] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:03 2022] PUT /identity/v3/projects/f7a54d85c0a5472d9c96613856e7c586/users/356d5836e1b347e0a7ffcd981b8347be/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:03.751244 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364cb8a-d2f1-45f5-b375-ec6026ba3c55 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:03.762035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e092c3e-cbc5-48e6-b023-72f0ac9c29b0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:03.765015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9e092c3e-cbc5-48e6-b023-72f0ac9c29b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:03.766488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364cb8a-d2f1-45f5-b375-ec6026ba3c55 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:03.848295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc785de2-40a8-45e5-b0e8-5032c14e7226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:03.849963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc785de2-40a8-45e5-b0e8-5032c14e7226 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:03.850452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc785de2-40a8-45e5-b0e8-5032c14e7226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:03.850925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc785de2-40a8-45e5-b0e8-5032c14e7226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:03.852832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc785de2-40a8-45e5-b0e8-5032c14e7226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:03.855990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc785de2-40a8-45e5-b0e8-5032c14e7226 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:03.872684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 896/1814] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:03.878349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc785de2-40a8-45e5-b0e8-5032c14e7226 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:03.913674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc785de2-40a8-45e5-b0e8-5032c14e7226 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:03.919589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c40797-22eb-4f26-926c-77068ee1663e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:03.920634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c40797-22eb-4f26-926c-77068ee1663e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:03.920936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c40797-22eb-4f26-926c-77068ee1663e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:03.921214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c40797-22eb-4f26-926c-77068ee1663e admin admin] PATH_INFO: `/v3/projects/f7a54d85c0a5472d9c96613856e7c586/users/356d5836e1b347e0a7ffcd981b8347be/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:03.927919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c40797-22eb-4f26-926c-77068ee1663e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a54d85c0a5472d9c96613856e7c586, user_id=356d5836e1b347e0a7ffcd981b8347be, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:03.930992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c40797-22eb-4f26-926c-77068ee1663e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:03.952813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 897/1815] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:03 2022] PUT /identity/v3/projects/f7a54d85c0a5472d9c96613856e7c586/users/356d5836e1b347e0a7ffcd981b8347be/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:03.958537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1bbb9-0020-486b-968f-3f8622743feb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:03.958937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1bbb9-0020-486b-968f-3f8622743feb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:03.959314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1bbb9-0020-486b-968f-3f8622743feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:04.005209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e092c3e-cbc5-48e6-b023-72f0ac9c29b0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:04.015245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5ad1bbb9-0020-486b-968f-3f8622743feb None None] MFA Rules not processed for user `356d5836e1b347e0a7ffcd981b8347be`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:04.017072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5ad1bbb9-0020-486b-968f-3f8622743feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:04.124973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 898/1816] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:04.129552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6b069-ff86-4c9c-bbd1-4568c4ae5758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:04.130640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6b069-ff86-4c9c-bbd1-4568c4ae5758 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:04.131105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6b069-ff86-4c9c-bbd1-4568c4ae5758 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:04.131518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6b069-ff86-4c9c-bbd1-4568c4ae5758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:04.133867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1fd6b069-ff86-4c9c-bbd1-4568c4ae5758 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:04.134511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1bbb9-0020-486b-968f-3f8622743feb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:04.135966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5ad1bbb9-0020-486b-968f-3f8622743feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:04.151284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1bbb9-0020-486b-968f-3f8622743feb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:04.157596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e6c26-27c3-4a19-95b2-f6612acc2ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:04.160632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e6c26-27c3-4a19-95b2-f6612acc2ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:04.160885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e6c26-27c3-4a19-95b2-f6612acc2ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:04.161426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e6c26-27c3-4a19-95b2-f6612acc2ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:04.193527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e6c26-27c3-4a19-95b2-f6612acc2ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:04.197566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e6c26-27c3-4a19-95b2-f6612acc2ad2 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:04.217601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:04.224041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 899/1817] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:36:04.254636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6b069-ff86-4c9c-bbd1-4568c4ae5758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:04.256956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6b069-ff86-4c9c-bbd1-4568c4ae5758 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:04.279840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 919/1818] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:05.882260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e6c26-27c3-4a19-95b2-f6612acc2ad2 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:05.888192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:05.981058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e6c26-27c3-4a19-95b2-f6612acc2ad2 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:05.986727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:05.987917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:05.989195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:05.992181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] PATH_INFO: `/v3/users/6dca49ea23144f739f07fc0148ee92a8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:05.993894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dca49ea23144f739f07fc0148ee92a8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:05.999073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:06.083686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dca49ea23144f739f07fc0148ee92a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:06.100552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dca49ea23144f739f07fc0148ee92a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:06.112709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dca49ea23144f739f07fc0148ee92a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:06.123585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dca49ea23144f739f07fc0148ee92a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:06.124546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dca49ea23144f739f07fc0148ee92a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:06.132457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 900/1819] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:05 2022] DELETE /identity/v3/users/6dca49ea23144f739f07fc0148ee92a8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:06.139499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f742ee-59fa-446f-ac95-771f237d3f7b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:06.140677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f742ee-59fa-446f-ac95-771f237d3f7b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:06.141073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f742ee-59fa-446f-ac95-771f237d3f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:06.160199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e0f742ee-59fa-446f-ac95-771f237d3f7b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:06.161481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e0f742ee-59fa-446f-ac95-771f237d3f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:06.252516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 920/1820] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:06.311541 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:06.313360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:06.447371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fdf40-9017-4ba9-ba45-8ab50deb33a0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:06.452003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2771f69-9e7f-4e90-94f8-a1ae9699ca8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:06.453459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2771f69-9e7f-4e90-94f8-a1ae9699ca8b service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:06.453459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2771f69-9e7f-4e90-94f8-a1ae9699ca8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:06.453459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2771f69-9e7f-4e90-94f8-a1ae9699ca8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:06.466098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2771f69-9e7f-4e90-94f8-a1ae9699ca8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:06.467578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2771f69-9e7f-4e90-94f8-a1ae9699ca8b service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:06.479284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 901/1821] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:06.987321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f742ee-59fa-446f-ac95-771f237d3f7b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:06.988822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e0f742ee-59fa-446f-ac95-771f237d3f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:07.019762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f742ee-59fa-446f-ac95-771f237d3f7b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:07.025899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:07.027032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:07.027345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:07.027813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] PATH_INFO: `/v3/projects/c9525c975f4e4f0d8327fd35a9d40601` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:07.030121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9525c975f4e4f0d8327fd35a9d40601)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:07.031633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:07.104147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9525c975f4e4f0d8327fd35a9d40601', 'request_id': 'req-957e992d-2725-4453-b879-8848874d1ee2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.113089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9525c975f4e4f0d8327fd35a9d40601', 'request_id': 'req-957e992d-2725-4453-b879-8848874d1ee2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.120414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9525c975f4e4f0d8327fd35a9d40601', 'request_id': 'req-957e992d-2725-4453-b879-8848874d1ee2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.133286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] The token cache is being invalidate because project c9525c975f4e4f0d8327fd35a9d40601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:07.133864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.134696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.136475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 921/1822] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:06 2022] DELETE /identity/v3/projects/c9525c975f4e4f0d8327fd35a9d40601 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:07.140515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2771f69-9e7f-4e90-94f8-a1ae9699ca8b service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:07.151554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2771f69-9e7f-4e90-94f8-a1ae9699ca8b service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:07.179928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:07.218814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:07.219702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:07.219951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:07.220286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] PATH_INFO: `/v3/users/356d5836e1b347e0a7ffcd981b8347be` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:07.221968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=356d5836e1b347e0a7ffcd981b8347be)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:07.224088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:07.284112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '356d5836e1b347e0a7ffcd981b8347be'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.296349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '356d5836e1b347e0a7ffcd981b8347be'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.305548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '356d5836e1b347e0a7ffcd981b8347be'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.312256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '356d5836e1b347e0a7ffcd981b8347be'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.312883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85d2e7cc-59c4-4a98-a0bc-e42b13467f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '356d5836e1b347e0a7ffcd981b8347be'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.319142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 902/1823] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:07 2022] DELETE /identity/v3/users/356d5836e1b347e0a7ffcd981b8347be => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:07.613893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:07.615345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:07.634903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e992d-2725-4453-b879-8848874d1ee2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:07.722847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:07.724118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:07.724358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:07.724700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] PATH_INFO: `/v3/projects/f7a54d85c0a5472d9c96613856e7c586` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:07.726389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a54d85c0a5472d9c96613856e7c586)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:07.728417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:07.737432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:07.787691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a54d85c0a5472d9c96613856e7c586', 'request_id': 'req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.798147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a54d85c0a5472d9c96613856e7c586', 'request_id': 'req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.804368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a54d85c0a5472d9c96613856e7c586', 'request_id': 'req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.826136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] The token cache is being invalidate because project f7a54d85c0a5472d9c96613856e7c586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:07.826442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.827162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40e4e00-4b4a-4490-a4b7-4beeaedd06ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:07.828804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 922/1824] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:07 2022] DELETE /identity/v3/projects/f7a54d85c0a5472d9c96613856e7c586 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:07.833912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659adf7e-ad27-4b8a-aaff-a508bbc14c66 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:07.834267 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659adf7e-ad27-4b8a-aaff-a508bbc14c66 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:07.834660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659adf7e-ad27-4b8a-aaff-a508bbc14c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:07.860342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-659adf7e-ad27-4b8a-aaff-a508bbc14c66 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:07.861859 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-659adf7e-ad27-4b8a-aaff-a508bbc14c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:07.938277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 903/1825] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:07.946253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdf067-3149-44f9-9aaa-105dd80b406a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:07.946539 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdf067-3149-44f9-9aaa-105dd80b406a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:07.946819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdf067-3149-44f9-9aaa-105dd80b406a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:07.965175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0bcdf067-3149-44f9-9aaa-105dd80b406a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:07.966376 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0bcdf067-3149-44f9-9aaa-105dd80b406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:07.974974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 923/1826] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:07.979227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659adf7e-ad27-4b8a-aaff-a508bbc14c66 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:07.989290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659adf7e-ad27-4b8a-aaff-a508bbc14c66 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:07.993832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc365d-1cc4-498e-b6e8-cf31411e6980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:07.994475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc365d-1cc4-498e-b6e8-cf31411e6980 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:07.994754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc365d-1cc4-498e-b6e8-cf31411e6980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:07.995026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc365d-1cc4-498e-b6e8-cf31411e6980 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:07.995759 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc365d-1cc4-498e-b6e8-cf31411e6980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:07.996063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc365d-1cc4-498e-b6e8-cf31411e6980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:07.997683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc365d-1cc4-498e-b6e8-cf31411e6980 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.009641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 904/1827] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:08.014019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdf067-3149-44f9-9aaa-105dd80b406a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.023566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdf067-3149-44f9-9aaa-105dd80b406a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.028596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cf5ef-a9df-40da-8a4d-5c7a3c003d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.029474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cf5ef-a9df-40da-8a4d-5c7a3c003d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.029762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cf5ef-a9df-40da-8a4d-5c7a3c003d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.030043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cf5ef-a9df-40da-8a4d-5c7a3c003d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.030772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31cf5ef-a9df-40da-8a4d-5c7a3c003d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.032745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31cf5ef-a9df-40da-8a4d-5c7a3c003d8d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.060438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 924/1828] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:08 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:36:08.065566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc365d-1cc4-498e-b6e8-cf31411e6980 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.076063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc365d-1cc4-498e-b6e8-cf31411e6980 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.111677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:08.145361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3a867-ba79-455d-8da5-e972396be1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.146037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3a867-ba79-455d-8da5-e972396be1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.146364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3a867-ba79-455d-8da5-e972396be1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.146648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3a867-ba79-455d-8da5-e972396be1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.147353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3a867-ba79-455d-8da5-e972396be1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.149337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3a867-ba79-455d-8da5-e972396be1f6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.169508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 905/1829] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:08 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:36:08.173712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cf5ef-a9df-40da-8a4d-5c7a3c003d8d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.185108 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cf5ef-a9df-40da-8a4d-5c7a3c003d8d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.191926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc371-a572-430a-afcb-3eff3f32e1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.192608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc371-a572-430a-afcb-3eff3f32e1db admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.192956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc371-a572-430a-afcb-3eff3f32e1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.193239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc371-a572-430a-afcb-3eff3f32e1db admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.194102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecc371-a572-430a-afcb-3eff3f32e1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.195895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecc371-a572-430a-afcb-3eff3f32e1db admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.209945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 925/1830] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:08.213862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3a867-ba79-455d-8da5-e972396be1f6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.224580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3a867-ba79-455d-8da5-e972396be1f6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.229163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0f835-b9a4-4da4-bee8-f19775627a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.229803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0f835-b9a4-4da4-bee8-f19775627a35 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.230088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0f835-b9a4-4da4-bee8-f19775627a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.230362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0f835-b9a4-4da4-bee8-f19775627a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.231089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0f835-b9a4-4da4-bee8-f19775627a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.232713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0f835-b9a4-4da4-bee8-f19775627a35 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.244613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 906/1831] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:08.249152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc371-a572-430a-afcb-3eff3f32e1db admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.260549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc371-a572-430a-afcb-3eff3f32e1db admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.264858 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15def88-0d92-4700-a158-764e02249094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.265804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15def88-0d92-4700-a158-764e02249094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.266052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15def88-0d92-4700-a158-764e02249094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.266329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15def88-0d92-4700-a158-764e02249094 admin admin] PATH_INFO: `/v3/projects/bf7c2e9a275d467e85faab4de4f56769/users/e90080a6c4f74f9487660ba5b74ec27c/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.284363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15def88-0d92-4700-a158-764e02249094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf7c2e9a275d467e85faab4de4f56769, user_id=e90080a6c4f74f9487660ba5b74ec27c, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.287944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15def88-0d92-4700-a158-764e02249094 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.304699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 926/1832] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:08 2022] PUT /identity/v3/projects/bf7c2e9a275d467e85faab4de4f56769/users/e90080a6c4f74f9487660ba5b74ec27c/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:08.316049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b117a2-dff9-4b3a-ade6-521f33f38d7e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.316415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b117a2-dff9-4b3a-ade6-521f33f38d7e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.316803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b117a2-dff9-4b3a-ade6-521f33f38d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.344706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f8b117a2-dff9-4b3a-ade6-521f33f38d7e None None] MFA Rules not processed for user `e90080a6c4f74f9487660ba5b74ec27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:08.346144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f8b117a2-dff9-4b3a-ade6-521f33f38d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:08.409921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 907/1833] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:08.414727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15def88-0d92-4700-a158-764e02249094 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.478942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15def88-0d92-4700-a158-764e02249094 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.483876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbc99b-0605-4051-961d-c88cf56f6b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.484473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbc99b-0605-4051-961d-c88cf56f6b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.484767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbc99b-0605-4051-961d-c88cf56f6b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.485065 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbc99b-0605-4051-961d-c88cf56f6b57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.485785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbc99b-0605-4051-961d-c88cf56f6b57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.488037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbc99b-0605-4051-961d-c88cf56f6b57 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.515432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 927/1834] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:08 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:08.519650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b117a2-dff9-4b3a-ade6-521f33f38d7e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.530892 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b117a2-dff9-4b3a-ade6-521f33f38d7e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.590818 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc696b2-ebce-4a00-b0f2-24e237711af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.591492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc696b2-ebce-4a00-b0f2-24e237711af6 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.591777 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc696b2-ebce-4a00-b0f2-24e237711af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.592058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc696b2-ebce-4a00-b0f2-24e237711af6 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.592783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc696b2-ebce-4a00-b0f2-24e237711af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.594701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc696b2-ebce-4a00-b0f2-24e237711af6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.616303 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 908/1835] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:08 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:08.620418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbc99b-0605-4051-961d-c88cf56f6b57 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.631198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbc99b-0605-4051-961d-c88cf56f6b57 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.636719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37392a11-fe69-429f-9b59-433308d5c167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.637417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37392a11-fe69-429f-9b59-433308d5c167 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.637705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37392a11-fe69-429f-9b59-433308d5c167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.637978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37392a11-fe69-429f-9b59-433308d5c167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.638709 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37392a11-fe69-429f-9b59-433308d5c167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.640379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37392a11-fe69-429f-9b59-433308d5c167 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.655825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 928/1836] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:08.662531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc696b2-ebce-4a00-b0f2-24e237711af6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.686533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc696b2-ebce-4a00-b0f2-24e237711af6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.691829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e22e1f-d416-44fd-9ca4-60af69c4deaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.692986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e22e1f-d416-44fd-9ca4-60af69c4deaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.693261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e22e1f-d416-44fd-9ca4-60af69c4deaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.693540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e22e1f-d416-44fd-9ca4-60af69c4deaf admin admin] PATH_INFO: `/v3/projects/c760dd41d33d44768ce5c0d50677506f/users/c3d0eb7e52304fffa2f768c4d3005a9e/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.715544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e22e1f-d416-44fd-9ca4-60af69c4deaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c760dd41d33d44768ce5c0d50677506f, user_id=c3d0eb7e52304fffa2f768c4d3005a9e, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.718316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e22e1f-d416-44fd-9ca4-60af69c4deaf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.738263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 909/1837] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:08 2022] PUT /identity/v3/projects/c760dd41d33d44768ce5c0d50677506f/users/c3d0eb7e52304fffa2f768c4d3005a9e/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:08.748768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37392a11-fe69-429f-9b59-433308d5c167 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.760428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37392a11-fe69-429f-9b59-433308d5c167 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.767757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:08.885641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ca72-0b66-4e6d-a484-6e93705cd66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.887078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ca72-0b66-4e6d-a484-6e93705cd66d admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.887424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ca72-0b66-4e6d-a484-6e93705cd66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.887815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ca72-0b66-4e6d-a484-6e93705cd66d admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.889104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79ca72-0b66-4e6d-a484-6e93705cd66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.891043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79ca72-0b66-4e6d-a484-6e93705cd66d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.915753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e22e1f-d416-44fd-9ca4-60af69c4deaf admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:08.928175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 929/1838] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:08.931523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e22e1f-d416-44fd-9ca4-60af69c4deaf admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:08.939875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1af64-cd10-460e-9ade-0818924213d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:08.941240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1af64-cd10-460e-9ade-0818924213d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.941520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1af64-cd10-460e-9ade-0818924213d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.941793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1af64-cd10-460e-9ade-0818924213d9 admin admin] PATH_INFO: `/v3/projects/c760dd41d33d44768ce5c0d50677506f/users/c3d0eb7e52304fffa2f768c4d3005a9e/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.943920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1af64-cd10-460e-9ade-0818924213d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c760dd41d33d44768ce5c0d50677506f, user_id=c3d0eb7e52304fffa2f768c4d3005a9e, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:08.948389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1af64-cd10-460e-9ade-0818924213d9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:08.950275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:08.974923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7ee09-756e-404a-8471-a042d98982d6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:08.975314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7ee09-756e-404a-8471-a042d98982d6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:08.976122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7ee09-756e-404a-8471-a042d98982d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:08.977653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 910/1839] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:08 2022] PUT /identity/v3/projects/c760dd41d33d44768ce5c0d50677506f/users/c3d0eb7e52304fffa2f768c4d3005a9e/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:09.018550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6db7ee09-756e-404a-8471-a042d98982d6 None None] MFA Rules not processed for user `c3d0eb7e52304fffa2f768c4d3005a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:09.020071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6db7ee09-756e-404a-8471-a042d98982d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:09.180224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 930/1840] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:09.189029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1af64-cd10-460e-9ade-0818924213d9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:09.191045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fad1af64-cd10-460e-9ade-0818924213d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:09.278706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1af64-cd10-460e-9ade-0818924213d9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:09.283194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5a3c5-1a6e-42da-9b4c-93a75afc775f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:09.283794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5a3c5-1a6e-42da-9b4c-93a75afc775f service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:09.284097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5a3c5-1a6e-42da-9b4c-93a75afc775f service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:09.285994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5a3c5-1a6e-42da-9b4c-93a75afc775f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:09.324350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5a3c5-1a6e-42da-9b4c-93a75afc775f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:09.325828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5a3c5-1a6e-42da-9b4c-93a75afc775f service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:09.340951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 911/1841] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:09.381662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7ee09-756e-404a-8471-a042d98982d6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:09.521624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7ee09-756e-404a-8471-a042d98982d6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:09.526190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:09.527247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:09.527377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:09.527565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] PATH_INFO: `/v3/users/e90080a6c4f74f9487660ba5b74ec27c` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:09.528849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e90080a6c4f74f9487660ba5b74ec27c)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:09.530799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:09.579980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:09.615499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e90080a6c4f74f9487660ba5b74ec27c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:09.642782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e90080a6c4f74f9487660ba5b74ec27c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:09.654898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e90080a6c4f74f9487660ba5b74ec27c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:09.655713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e90080a6c4f74f9487660ba5b74ec27c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:09.669722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e90080a6c4f74f9487660ba5b74ec27c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:09.678224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 931/1842] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:09 2022] DELETE /identity/v3/users/e90080a6c4f74f9487660ba5b74ec27c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:09.682701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed0b4b-7915-435f-91d0-f840404f3f81 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:09.682981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed0b4b-7915-435f-91d0-f840404f3f81 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:09.683263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed0b4b-7915-435f-91d0-f840404f3f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:09.702965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e7ed0b4b-7915-435f-91d0-f840404f3f81 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:09.704160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e7ed0b4b-7915-435f-91d0-f840404f3f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:09.775418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 912/1843] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:09.785838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:09.787646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:09.855817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed0b4b-7915-435f-91d0-f840404f3f81 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:09.857279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e7ed0b4b-7915-435f-91d0-f840404f3f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:09.862460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96dfdd-8a2e-4892-a9b7-5bb01db0c579 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:09.866838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929358f9-ee29-4e4a-9065-713221a329ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:09.867569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929358f9-ee29-4e4a-9065-713221a329ed service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:09.867890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929358f9-ee29-4e4a-9065-713221a329ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:09.868348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929358f9-ee29-4e4a-9065-713221a329ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:09.876432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed0b4b-7915-435f-91d0-f840404f3f81 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:09.881296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:09.882218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:09.882502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:09.882777 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] PATH_INFO: `/v3/users/79b4055ead3049e4a14892397c829c78` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:09.885028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b4055ead3049e4a14892397c829c78)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:09.885545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929358f9-ee29-4e4a-9065-713221a329ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:09.887856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929358f9-ee29-4e4a-9065-713221a329ed service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:09.887963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:09.900813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 932/1844] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:09.954076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b4055ead3049e4a14892397c829c78'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:09.973353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b4055ead3049e4a14892397c829c78'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:09.985931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b4055ead3049e4a14892397c829c78'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:09.994976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b4055ead3049e4a14892397c829c78'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:09.995754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b4055ead3049e4a14892397c829c78'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.003580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 913/1845] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:09 2022] DELETE /identity/v3/users/79b4055ead3049e4a14892397c829c78 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:10.008302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179c111-94e7-4484-8b0b-005e492c5d43 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:10.008564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179c111-94e7-4484-8b0b-005e492c5d43 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:10.008871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179c111-94e7-4484-8b0b-005e492c5d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:10.027619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2179c111-94e7-4484-8b0b-005e492c5d43 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:10.028882 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2179c111-94e7-4484-8b0b-005e492c5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:10.101148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 933/1846] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:10.110562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:10.112487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:10.112791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179c111-94e7-4484-8b0b-005e492c5d43 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:10.114386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2179c111-94e7-4484-8b0b-005e492c5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:10.125333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:10.126352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179c111-94e7-4484-8b0b-005e492c5d43 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:10.133554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:10.134587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:10.134935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:10.136172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] PATH_INFO: `/v3/users/aeb122f4467347dc97bc2694e318b81d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:10.138716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb122f4467347dc97bc2694e318b81d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:10.142058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:10.201837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbba47-e338-4eeb-a149-72ccd668fe7b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:10.202932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:10.205993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb122f4467347dc97bc2694e318b81d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.209527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c2050-bd69-40da-b946-e86db8867f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:10.210807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2050-bd69-40da-b946-e86db8867f4f service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:10.211324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2050-bd69-40da-b946-e86db8867f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:10.212027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2050-bd69-40da-b946-e86db8867f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:10.226679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb122f4467347dc97bc2694e318b81d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.228803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c2050-bd69-40da-b946-e86db8867f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:10.231803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c2050-bd69-40da-b946-e86db8867f4f service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:10.238247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb122f4467347dc97bc2694e318b81d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.240579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb122f4467347dc97bc2694e318b81d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.247080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 914/1847] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:10.252431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb122f4467347dc97bc2694e318b81d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.260033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 934/1848] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:10 2022] DELETE /identity/v3/users/aeb122f4467347dc97bc2694e318b81d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:10.264181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155444fa-18c0-41d1-899c-a157e229363f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:10.265082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155444fa-18c0-41d1-899c-a157e229363f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:10.265980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155444fa-18c0-41d1-899c-a157e229363f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:10.285955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-155444fa-18c0-41d1-899c-a157e229363f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:10.287221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-155444fa-18c0-41d1-899c-a157e229363f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:10.402396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 915/1849] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:10.445174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:10.447007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:10.535280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0215d-badb-4992-81e9-38fe93a4833b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:10.539302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11516154-0a47-40fe-ab2d-164345611d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:10.540005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11516154-0a47-40fe-ab2d-164345611d40 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:10.540292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11516154-0a47-40fe-ab2d-164345611d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:10.540573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11516154-0a47-40fe-ab2d-164345611d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:10.552676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11516154-0a47-40fe-ab2d-164345611d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:10.554196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11516154-0a47-40fe-ab2d-164345611d40 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:10.566049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 935/1850] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:10.770499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155444fa-18c0-41d1-899c-a157e229363f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:10.772040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-155444fa-18c0-41d1-899c-a157e229363f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:10.795599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155444fa-18c0-41d1-899c-a157e229363f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:10.802261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:10.803603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:10.804090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:10.804565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] PATH_INFO: `/v3/projects/bf7c2e9a275d467e85faab4de4f56769` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:10.806080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf7c2e9a275d467e85faab4de4f56769)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:10.808436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:10.892714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf7c2e9a275d467e85faab4de4f56769', 'request_id': 'req-43009108-56ce-41c0-b6a4-6f9bf6dd7051'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.902090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf7c2e9a275d467e85faab4de4f56769', 'request_id': 'req-43009108-56ce-41c0-b6a4-6f9bf6dd7051'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.922349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf7c2e9a275d467e85faab4de4f56769', 'request_id': 'req-43009108-56ce-41c0-b6a4-6f9bf6dd7051'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.940711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] The token cache is being invalidate because project bf7c2e9a275d467e85faab4de4f56769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:10.941040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.945039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-43009108-56ce-41c0-b6a4-6f9bf6dd7051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:10.949107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 916/1851] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:10 2022] DELETE /identity/v3/projects/bf7c2e9a275d467e85faab4de4f56769 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:10.952666 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11516154-0a47-40fe-ab2d-164345611d40 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:10.970149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11516154-0a47-40fe-ab2d-164345611d40 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:11.083160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:11.084473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:11.085011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:11.085484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] PATH_INFO: `/v3/users/c3d0eb7e52304fffa2f768c4d3005a9e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:11.087899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3d0eb7e52304fffa2f768c4d3005a9e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:11.090380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:11.167423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3d0eb7e52304fffa2f768c4d3005a9e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.184875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3d0eb7e52304fffa2f768c4d3005a9e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.188776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:11.203926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3d0eb7e52304fffa2f768c4d3005a9e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.204919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3d0eb7e52304fffa2f768c4d3005a9e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.206512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca065b-9ec9-4e83-8c7c-b95599b1ad07 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:11.207377 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca065b-9ec9-4e83-8c7c-b95599b1ad07 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:11.207830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca065b-9ec9-4e83-8c7c-b95599b1ad07 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:11.210580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 917/1852] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:36:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:36:11.224203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3d0eb7e52304fffa2f768c4d3005a9e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.232944 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a70a-74f9-47ec-90ab-292ba6fca6c2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:11.233411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a70a-74f9-47ec-90ab-292ba6fca6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:11.233787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a70a-74f9-47ec-90ab-292ba6fca6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:11.236240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 936/1853] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:10 2022] DELETE /identity/v3/users/c3d0eb7e52304fffa2f768c4d3005a9e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:11.288474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-07f1a70a-74f9-47ec-90ab-292ba6fca6c2 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:11.289821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-07f1a70a-74f9-47ec-90ab-292ba6fca6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:11.343321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:11.397864 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 918/1854] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:11.410227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:11.412576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:11.426982 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a70a-74f9-47ec-90ab-292ba6fca6c2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:11.431843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-07f1a70a-74f9-47ec-90ab-292ba6fca6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:11.456402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a70a-74f9-47ec-90ab-292ba6fca6c2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:11.467302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea88ab2-f8a1-4f6a-bcc1-cf9e217c40bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:11.468103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea88ab2-f8a1-4f6a-bcc1-cf9e217c40bb service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:11.468559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea88ab2-f8a1-4f6a-bcc1-cf9e217c40bb service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:11.469402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea88ab2-f8a1-4f6a-bcc1-cf9e217c40bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:11.482443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea88ab2-f8a1-4f6a-bcc1-cf9e217c40bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:11.484728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea88ab2-f8a1-4f6a-bcc1-cf9e217c40bb service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:11.510958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 919/1855] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:36:11.536578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cf8bc-07ae-4d23-ae94-892805e4e282 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:11.541450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:11.542435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:11.542685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:11.543013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] PATH_INFO: `/v3/projects/021f1bf5e16b457487b35b5a3eee6441` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:11.544558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=021f1bf5e16b457487b35b5a3eee6441)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:11.546580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:11.577592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea88ab2-f8a1-4f6a-bcc1-cf9e217c40bb service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:11.580214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ea88ab2-f8a1-4f6a-bcc1-cf9e217c40bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:11.609777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021f1bf5e16b457487b35b5a3eee6441', 'request_id': 'req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.620300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea88ab2-f8a1-4f6a-bcc1-cf9e217c40bb service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:11.621089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021f1bf5e16b457487b35b5a3eee6441', 'request_id': 'req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.629330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021f1bf5e16b457487b35b5a3eee6441', 'request_id': 'req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.646493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] The token cache is being invalidate because project 021f1bf5e16b457487b35b5a3eee6441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:11.646837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.647648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e2c5d6-e36e-4ebc-b18a-0cc1bdbd24cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.649821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 937/1856] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:11 2022] DELETE /identity/v3/projects/021f1bf5e16b457487b35b5a3eee6441 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:11.658078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211728b-4854-42dc-a876-913eea788ae7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:11.658078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211728b-4854-42dc-a876-913eea788ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:11.658854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211728b-4854-42dc-a876-913eea788ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:11.680455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d211728b-4854-42dc-a876-913eea788ae7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:11.681765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d211728b-4854-42dc-a876-913eea788ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:11.693467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:11.694384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:11.694874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:11.695098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] PATH_INFO: `/v3/projects/810b447768eb4c5f84b81078a79766bb` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:11.696308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=810b447768eb4c5f84b81078a79766bb)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:11.698381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:11.759506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810b447768eb4c5f84b81078a79766bb', 'request_id': 'req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.776409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810b447768eb4c5f84b81078a79766bb', 'request_id': 'req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.786442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 938/1857] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:11.788873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211728b-4854-42dc-a876-913eea788ae7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:11.791711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d211728b-4854-42dc-a876-913eea788ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:11.797102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810b447768eb4c5f84b81078a79766bb', 'request_id': 'req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.812825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] The token cache is being invalidate because project 810b447768eb4c5f84b81078a79766bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:11.813120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.813950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0ace44f-2e2f-4085-b808-ac93d15ce2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:11.815185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211728b-4854-42dc-a876-913eea788ae7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:11.815842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 920/1858] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:11 2022] DELETE /identity/v3/projects/810b447768eb4c5f84b81078a79766bb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:11.817214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02886e-45ab-4ce6-ab7f-1cc5264471dc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:11.817503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02886e-45ab-4ce6-ab7f-1cc5264471dc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:11.817785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02886e-45ab-4ce6-ab7f-1cc5264471dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:11.835401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:11.913299 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:11.914540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:11.914979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:11.915581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] PATH_INFO: `/v3/projects/c760dd41d33d44768ce5c0d50677506f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:11.917208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c760dd41d33d44768ce5c0d50677506f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:11.919410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.010198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c760dd41d33d44768ce5c0d50677506f', 'request_id': 'req-2802bdfc-29de-4672-9e5c-60c6f60816cc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.010847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8b02886e-45ab-4ce6-ab7f-1cc5264471dc None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:12.011755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8b02886e-45ab-4ce6-ab7f-1cc5264471dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:12.039430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c760dd41d33d44768ce5c0d50677506f', 'request_id': 'req-2802bdfc-29de-4672-9e5c-60c6f60816cc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.056253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c760dd41d33d44768ce5c0d50677506f', 'request_id': 'req-2802bdfc-29de-4672-9e5c-60c6f60816cc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.080308 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] The token cache is being invalidate because project c760dd41d33d44768ce5c0d50677506f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:12.082162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.085813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.088287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 939/1859] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:11 2022] DELETE /identity/v3/projects/c760dd41d33d44768ce5c0d50677506f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:12.092137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.097138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:12.099934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:12.138287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 921/1860] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:12.140640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0060d1-f8b9-4204-8082-0d7f614d5d5c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.142151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0060d1-f8b9-4204-8082-0d7f614d5d5c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.142544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0060d1-f8b9-4204-8082-0d7f614d5d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.163744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5d0060d1-f8b9-4204-8082-0d7f614d5d5c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:12.165167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5d0060d1-f8b9-4204-8082-0d7f614d5d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:12.180132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 922/1861] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:12.181645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0060d1-f8b9-4204-8082-0d7f614d5d5c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.198743 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0060d1-f8b9-4204-8082-0d7f614d5d5c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.206474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40531735-6d4c-45bd-a667-787b489b6a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.207301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531735-6d4c-45bd-a667-787b489b6a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.207851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531735-6d4c-45bd-a667-787b489b6a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.208198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531735-6d4c-45bd-a667-787b489b6a06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.209080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40531735-6d4c-45bd-a667-787b489b6a06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:12.209804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40531735-6d4c-45bd-a667-787b489b6a06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.211695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40531735-6d4c-45bd-a667-787b489b6a06 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.219125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802bdfc-29de-4672-9e5c-60c6f60816cc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.224228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.225430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.225711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.225985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] PATH_INFO: `/v3/users/6cef4fae28f54171b9839d65393c8f32` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.227206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cef4fae28f54171b9839d65393c8f32)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.228141 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 923/1862] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:12.229948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f8f65-deda-48df-b9fa-e44670db405e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.230345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f8f65-deda-48df-b9fa-e44670db405e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.230740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f8f65-deda-48df-b9fa-e44670db405e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.232996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.255446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3d5f8f65-deda-48df-b9fa-e44670db405e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:12.256915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3d5f8f65-deda-48df-b9fa-e44670db405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:12.268240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 924/1863] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:12.270059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f8f65-deda-48df-b9fa-e44670db405e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.284101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f8f65-deda-48df-b9fa-e44670db405e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.291596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb8ba3-d1b9-4754-a666-519c06e2ea6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.292500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb8ba3-d1b9-4754-a666-519c06e2ea6c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.292880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb8ba3-d1b9-4754-a666-519c06e2ea6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.293238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb8ba3-d1b9-4754-a666-519c06e2ea6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.294453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb8ba3-d1b9-4754-a666-519c06e2ea6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.296555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb8ba3-d1b9-4754-a666-519c06e2ea6c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.310464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cef4fae28f54171b9839d65393c8f32'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.311844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:12.332625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 925/1864] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:12 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:12.333738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb8ba3-d1b9-4754-a666-519c06e2ea6c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.340914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cef4fae28f54171b9839d65393c8f32'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.352278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cef4fae28f54171b9839d65393c8f32'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.353256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cef4fae28f54171b9839d65393c8f32'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.365937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cef4fae28f54171b9839d65393c8f32'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:12.374489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 940/1865] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:12 2022] DELETE /identity/v3/users/6cef4fae28f54171b9839d65393c8f32 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:12.375542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.378262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:12.412720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb8ba3-d1b9-4754-a666-519c06e2ea6c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.466872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac470d16-9e1b-43b3-9ba1-48188ebfce1a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.474722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a6aaf-907f-4f9b-b9dd-2c47f1bc50f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.475751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a6aaf-907f-4f9b-b9dd-2c47f1bc50f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.476236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a6aaf-907f-4f9b-b9dd-2c47f1bc50f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.477355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a6aaf-907f-4f9b-b9dd-2c47f1bc50f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.478161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a6aaf-907f-4f9b-b9dd-2c47f1bc50f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.481043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.481517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a6aaf-907f-4f9b-b9dd-2c47f1bc50f8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.481865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.482237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.482585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.483402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:12.483781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.485744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.508194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 926/1866] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:12.512441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.514416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:12.528659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccc426-3bd0-40b1-8df4-83680409bb2f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.529847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 941/1867] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:12 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:12.536116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a6aaf-907f-4f9b-b9dd-2c47f1bc50f8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.537186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b50ce-2b62-4e12-b0f1-eb353d736587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.537999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b50ce-2b62-4e12-b0f1-eb353d736587 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.538508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b50ce-2b62-4e12-b0f1-eb353d736587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.538891 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b50ce-2b62-4e12-b0f1-eb353d736587 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.540062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b50ce-2b62-4e12-b0f1-eb353d736587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.542312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b50ce-2b62-4e12-b0f1-eb353d736587 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.552319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a6aaf-907f-4f9b-b9dd-2c47f1bc50f8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.559018 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4acc9-5dbc-47a8-bb38-6d8b819b76c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.559769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4acc9-5dbc-47a8-bb38-6d8b819b76c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.560372 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4acc9-5dbc-47a8-bb38-6d8b819b76c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.560664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4acc9-5dbc-47a8-bb38-6d8b819b76c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.561580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4acc9-5dbc-47a8-bb38-6d8b819b76c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.564335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4acc9-5dbc-47a8-bb38-6d8b819b76c4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.595132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 927/1868] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:12 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:12.595365 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 942/1869] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:12.600004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b50ce-2b62-4e12-b0f1-eb353d736587 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.601670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4acc9-5dbc-47a8-bb38-6d8b819b76c4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.624077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4acc9-5dbc-47a8-bb38-6d8b819b76c4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.651739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:12.759268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b50ce-2b62-4e12-b0f1-eb353d736587 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.775376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f8b5c-734b-4c3a-b8b8-8635347011f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.776567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f8b5c-734b-4c3a-b8b8-8635347011f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.776567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f8b5c-734b-4c3a-b8b8-8635347011f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.776981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f8b5c-734b-4c3a-b8b8-8635347011f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.777795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f8b5c-734b-4c3a-b8b8-8635347011f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.785723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f8b5c-734b-4c3a-b8b8-8635347011f7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.854572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 928/1870] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:12 2022] POST /identity/v3/users => generated 288 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:36:12.858737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f8b5c-734b-4c3a-b8b8-8635347011f7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.880072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f8b5c-734b-4c3a-b8b8-8635347011f7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.886124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620e2d2-3086-43b0-b373-4f72b7bd6996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.887052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055265-47ac-47d2-b574-578a669e4848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.887713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620e2d2-3086-43b0-b373-4f72b7bd6996 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.887937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055265-47ac-47d2-b574-578a669e4848 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.888047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620e2d2-3086-43b0-b373-4f72b7bd6996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.888341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055265-47ac-47d2-b574-578a669e4848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.888689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055265-47ac-47d2-b574-578a669e4848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.890098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620e2d2-3086-43b0-b373-4f72b7bd6996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.890532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055265-47ac-47d2-b574-578a669e4848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.892122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620e2d2-3086-43b0-b373-4f72b7bd6996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.893321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055265-47ac-47d2-b574-578a669e4848 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.896798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620e2d2-3086-43b0-b373-4f72b7bd6996 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.908466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 929/1871] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:12.913047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055265-47ac-47d2-b574-578a669e4848 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.920202 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 943/1872] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:12.924354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620e2d2-3086-43b0-b373-4f72b7bd6996 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.925223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055265-47ac-47d2-b574-578a669e4848 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.931044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e635536-98bf-46a7-b0a3-653e07fd43fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.931855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e635536-98bf-46a7-b0a3-653e07fd43fb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.932231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e635536-98bf-46a7-b0a3-653e07fd43fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.932572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e635536-98bf-46a7-b0a3-653e07fd43fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.933424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e635536-98bf-46a7-b0a3-653e07fd43fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:12.935157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e635536-98bf-46a7-b0a3-653e07fd43fb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:12.943962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620e2d2-3086-43b0-b373-4f72b7bd6996 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.949556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 930/1873] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:12.954644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e635536-98bf-46a7-b0a3-653e07fd43fb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:12.962347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f52d79-7521-48e3-a514-aed6407a7027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.963274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f52d79-7521-48e3-a514-aed6407a7027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.963633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f52d79-7521-48e3-a514-aed6407a7027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.963838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f52d79-7521-48e3-a514-aed6407a7027 admin admin] PATH_INFO: `/v3/projects/a5b92753e536433794bba24ae2e7c225/users/3720f34965344d72996ff684b8458804/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:12.977235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e635536-98bf-46a7-b0a3-653e07fd43fb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:12.988789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2add833-166c-49e5-a270-c0c2cd938236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:12.989716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2add833-166c-49e5-a270-c0c2cd938236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:12.990071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2add833-166c-49e5-a270-c0c2cd938236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:12.990369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2add833-166c-49e5-a270-c0c2cd938236 admin admin] PATH_INFO: `/v3/projects/4139f18358794e7b8ea7cb2486864847/users/363acb6f119a40eea2cf174fd9e09598/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.003431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f52d79-7521-48e3-a514-aed6407a7027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b92753e536433794bba24ae2e7c225, user_id=3720f34965344d72996ff684b8458804, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.020606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f52d79-7521-48e3-a514-aed6407a7027 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.037625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2add833-166c-49e5-a270-c0c2cd938236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4139f18358794e7b8ea7cb2486864847, user_id=363acb6f119a40eea2cf174fd9e09598, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.045296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2add833-166c-49e5-a270-c0c2cd938236 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.051152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 944/1874] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:12 2022] PUT /identity/v3/projects/a5b92753e536433794bba24ae2e7c225/users/3720f34965344d72996ff684b8458804/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:13.063711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b210c-5178-47c7-923d-35932143145e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.064338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b210c-5178-47c7-923d-35932143145e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.064812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b210c-5178-47c7-923d-35932143145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.088199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 931/1875] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:12 2022] PUT /identity/v3/projects/4139f18358794e7b8ea7cb2486864847/users/363acb6f119a40eea2cf174fd9e09598/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:13.105374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16268b3-5032-4f36-b91e-a723cd501ed8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.105605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16268b3-5032-4f36-b91e-a723cd501ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.105919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16268b3-5032-4f36-b91e-a723cd501ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.137745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-203b210c-5178-47c7-923d-35932143145e None None] MFA Rules not processed for user `3720f34965344d72996ff684b8458804`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:13.139000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-203b210c-5178-47c7-923d-35932143145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:13.148487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:13.149196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b16268b3-5032-4f36-b91e-a723cd501ed8 None None] MFA Rules not processed for user `363acb6f119a40eea2cf174fd9e09598`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:13.151884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b16268b3-5032-4f36-b91e-a723cd501ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:13.262229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 932/1876] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:13.263956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16268b3-5032-4f36-b91e-a723cd501ed8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:13.265889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b16268b3-5032-4f36-b91e-a723cd501ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:13.281195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 945/1877] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:13.286755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b210c-5178-47c7-923d-35932143145e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:13.330141 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:13.353806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16268b3-5032-4f36-b91e-a723cd501ed8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:13.364615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:13.368011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.368395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.368688 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] PATH_INFO: `/v3/projects/e3c9429879e3458aa5e7511e60e541c7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.372832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c9429879e3458aa5e7511e60e541c7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.378355 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.387392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b210c-5178-47c7-923d-35932143145e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:13.394598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53599ab0-14a5-49a3-bfec-d21009f4f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:13.396023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53599ab0-14a5-49a3-bfec-d21009f4f877 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.396023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53599ab0-14a5-49a3-bfec-d21009f4f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.396235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53599ab0-14a5-49a3-bfec-d21009f4f877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.396931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53599ab0-14a5-49a3-bfec-d21009f4f877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.399180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53599ab0-14a5-49a3-bfec-d21009f4f877 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.435653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 946/1878] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:13 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:13.436788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53599ab0-14a5-49a3-bfec-d21009f4f877 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:13.446785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-53599ab0-14a5-49a3-bfec-d21009f4f877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:13.462452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c9429879e3458aa5e7511e60e541c7', 'request_id': 'req-56b7d2dc-538a-4174-aaef-3a677dcd93de'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.473882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c9429879e3458aa5e7511e60e541c7', 'request_id': 'req-56b7d2dc-538a-4174-aaef-3a677dcd93de'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.484361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c9429879e3458aa5e7511e60e541c7', 'request_id': 'req-56b7d2dc-538a-4174-aaef-3a677dcd93de'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.514863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] The token cache is being invalidate because project e3c9429879e3458aa5e7511e60e541c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:13.514863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.514863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.514863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 933/1879] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:13 2022] DELETE /identity/v3/projects/e3c9429879e3458aa5e7511e60e541c7 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:13.518252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:13.522852 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:13.545007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53599ab0-14a5-49a3-bfec-d21009f4f877 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:13.589014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:13.591707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7d2dc-538a-4174-aaef-3a677dcd93de admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:13.596408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526afda4-2e55-4109-bc90-a4890311ffbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:13.596636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5290a61-4390-4b8c-a215-03d79386612d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:13.597427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526afda4-2e55-4109-bc90-a4890311ffbd service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.597540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5290a61-4390-4b8c-a215-03d79386612d admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.597754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5290a61-4390-4b8c-a215-03d79386612d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.597857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526afda4-2e55-4109-bc90-a4890311ffbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.598028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5290a61-4390-4b8c-a215-03d79386612d admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.598148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526afda4-2e55-4109-bc90-a4890311ffbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.598757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5290a61-4390-4b8c-a215-03d79386612d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.600112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-526afda4-2e55-4109-bc90-a4890311ffbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:13.600660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5290a61-4390-4b8c-a215-03d79386612d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.621952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 934/1880] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:13 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:13.625578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5290a61-4390-4b8c-a215-03d79386612d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:13.627444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b5290a61-4390-4b8c-a215-03d79386612d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:13.639096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5290a61-4390-4b8c-a215-03d79386612d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:13.643662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:13.644529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.644834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.645105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] PATH_INFO: `/v3/users/9fb8b53c0bf74db081f3a1ecf205f7a6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.646271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb8b53c0bf74db081f3a1ecf205f7a6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.648172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.663386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526afda4-2e55-4109-bc90-a4890311ffbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.664883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526afda4-2e55-4109-bc90-a4890311ffbd service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.674079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 947/1881] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:13.675048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526afda4-2e55-4109-bc90-a4890311ffbd service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:13.686517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526afda4-2e55-4109-bc90-a4890311ffbd service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:13.692676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4349e-ff8d-44bd-a6af-235907b908dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:13.693384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4349e-ff8d-44bd-a6af-235907b908dd admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.693670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4349e-ff8d-44bd-a6af-235907b908dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.693946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4349e-ff8d-44bd-a6af-235907b908dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.694678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4349e-ff8d-44bd-a6af-235907b908dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.696267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4349e-ff8d-44bd-a6af-235907b908dd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.703742 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb8b53c0bf74db081f3a1ecf205f7a6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.706867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 948/1882] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:13.710837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4349e-ff8d-44bd-a6af-235907b908dd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:13.716713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=80,keystone:DELETE=16,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:13.718182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb8b53c0bf74db081f3a1ecf205f7a6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.724111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4349e-ff8d-44bd-a6af-235907b908dd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:13.728264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb8b53c0bf74db081f3a1ecf205f7a6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.729420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553be2c-80fd-442a-811f-296d63399786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:13.730319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553be2c-80fd-442a-811f-296d63399786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.730609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553be2c-80fd-442a-811f-296d63399786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.730890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553be2c-80fd-442a-811f-296d63399786 admin admin] PATH_INFO: `/v3/projects/92f9a0d4561548f9becad44d0fca601b/users/7eea1231baa2408a9955a1dbab366d4e/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.737341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb8b53c0bf74db081f3a1ecf205f7a6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.745207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb8b53c0bf74db081f3a1ecf205f7a6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:13.747933 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553be2c-80fd-442a-811f-296d63399786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f9a0d4561548f9becad44d0fca601b, user_id=7eea1231baa2408a9955a1dbab366d4e, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.750686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553be2c-80fd-442a-811f-296d63399786 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.755308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 935/1883] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:13 2022] DELETE /identity/v3/users/9fb8b53c0bf74db081f3a1ecf205f7a6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:13.770760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 949/1884] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:13 2022] PUT /identity/v3/projects/92f9a0d4561548f9becad44d0fca601b/users/7eea1231baa2408a9955a1dbab366d4e/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:13.775101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:13.776870 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:13.857976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d09587-093f-4d3e-b3d5-a61335cc91cb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:13.964230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64985-130b-49a4-8d34-2701f8db0750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:13.965599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64985-130b-49a4-8d34-2701f8db0750 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:13.966079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64985-130b-49a4-8d34-2701f8db0750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:13.966421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64985-130b-49a4-8d34-2701f8db0750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:13.967703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f64985-130b-49a4-8d34-2701f8db0750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:13.970072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f64985-130b-49a4-8d34-2701f8db0750 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:13.984071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 936/1885] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:13.989180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553be2c-80fd-442a-811f-296d63399786 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:14.007522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553be2c-80fd-442a-811f-296d63399786 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:14.021171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bf2ad-5e1f-439d-8d5e-4aecf7e3fdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:14.022172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bf2ad-5e1f-439d-8d5e-4aecf7e3fdb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.022460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bf2ad-5e1f-439d-8d5e-4aecf7e3fdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.022740 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bf2ad-5e1f-439d-8d5e-4aecf7e3fdb7 admin admin] PATH_INFO: `/v3/projects/92f9a0d4561548f9becad44d0fca601b/users/7eea1231baa2408a9955a1dbab366d4e/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:14.035059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17bf2ad-5e1f-439d-8d5e-4aecf7e3fdb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f9a0d4561548f9becad44d0fca601b, user_id=7eea1231baa2408a9955a1dbab366d4e, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:14.041345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17bf2ad-5e1f-439d-8d5e-4aecf7e3fdb7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:14.058750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 950/1886] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:13 2022] PUT /identity/v3/projects/92f9a0d4561548f9becad44d0fca601b/users/7eea1231baa2408a9955a1dbab366d4e/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:14.070981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ec79f-3d5e-423d-b135-f93c6396e253 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.071351 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ec79f-3d5e-423d-b135-f93c6396e253 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.072217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ec79f-3d5e-423d-b135-f93c6396e253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:14.135518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5f0ec79f-3d5e-423d-b135-f93c6396e253 None None] MFA Rules not processed for user `7eea1231baa2408a9955a1dbab366d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:14.137715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5f0ec79f-3d5e-423d-b135-f93c6396e253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:14.221917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 937/1887] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:14.234281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bf2ad-5e1f-439d-8d5e-4aecf7e3fdb7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:14.237064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e17bf2ad-5e1f-439d-8d5e-4aecf7e3fdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:14.304956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bf2ad-5e1f-439d-8d5e-4aecf7e3fdb7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:14.309896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e53e1-636e-4f8e-b976-a856c46f85b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:14.310706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e53e1-636e-4f8e-b976-a856c46f85b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.311031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e53e1-636e-4f8e-b976-a856c46f85b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.311314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e53e1-636e-4f8e-b976-a856c46f85b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:14.312354 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ec79f-3d5e-423d-b135-f93c6396e253 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:14.314703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5f0ec79f-3d5e-423d-b135-f93c6396e253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:14.326026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e53e1-636e-4f8e-b976-a856c46f85b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:14.327815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e53e1-636e-4f8e-b976-a856c46f85b1 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:14.330780 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:14.346601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 951/1888] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:14.423896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ec79f-3d5e-423d-b135-f93c6396e253 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:14.430021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:14.430982 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.431261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.431553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] PATH_INFO: `/v3/projects/e120b72afd2a40f9b61154b62bcc00f0` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:14.432884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e120b72afd2a40f9b61154b62bcc00f0)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:14.434902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:14.503164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e120b72afd2a40f9b61154b62bcc00f0', 'request_id': 'req-2c20bc5f-c157-4fda-909c-a8690b8199b9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:14.511407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e120b72afd2a40f9b61154b62bcc00f0', 'request_id': 'req-2c20bc5f-c157-4fda-909c-a8690b8199b9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:14.536917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e120b72afd2a40f9b61154b62bcc00f0', 'request_id': 'req-2c20bc5f-c157-4fda-909c-a8690b8199b9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:14.559223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] The token cache is being invalidate because project e120b72afd2a40f9b61154b62bcc00f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:14.560357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:14.564662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c20bc5f-c157-4fda-909c-a8690b8199b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:14.568077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 938/1889] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:14 2022] DELETE /identity/v3/projects/e120b72afd2a40f9b61154b62bcc00f0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:14.575977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31712d-80aa-450c-9ac3-0152cf6a3dd3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.576280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31712d-80aa-450c-9ac3-0152cf6a3dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.576567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31712d-80aa-450c-9ac3-0152cf6a3dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:14.617843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4a31712d-80aa-450c-9ac3-0152cf6a3dd3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:14.619086 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4a31712d-80aa-450c-9ac3-0152cf6a3dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:14.683587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:14.699535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 952/1890] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:14.707857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515d46-7836-4e6f-8ade-052e36c93b61 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.708168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515d46-7836-4e6f-8ade-052e36c93b61 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.708440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515d46-7836-4e6f-8ade-052e36c93b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:14.729637 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e6515d46-7836-4e6f-8ade-052e36c93b61 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:14.731348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e6515d46-7836-4e6f-8ade-052e36c93b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:14.748951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 939/1891] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:14.753369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31712d-80aa-450c-9ac3-0152cf6a3dd3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:14.765618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31712d-80aa-450c-9ac3-0152cf6a3dd3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:14.774587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909016-aed6-48ec-bd7f-6610f8dc7502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:14.775244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909016-aed6-48ec-bd7f-6610f8dc7502 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.775577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909016-aed6-48ec-bd7f-6610f8dc7502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.775935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909016-aed6-48ec-bd7f-6610f8dc7502 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:14.776633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37909016-aed6-48ec-bd7f-6610f8dc7502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:14.776993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37909016-aed6-48ec-bd7f-6610f8dc7502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:14.778706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37909016-aed6-48ec-bd7f-6610f8dc7502 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:14.798430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6515d46-7836-4e6f-8ade-052e36c93b61 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:14.804824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 953/1892] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:14.808879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6515d46-7836-4e6f-8ade-052e36c93b61 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:14.814758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4ea6c-efd6-4eaf-8f5b-ee8ceba38aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:14.815480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4ea6c-efd6-4eaf-8f5b-ee8ceba38aea admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.815792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4ea6c-efd6-4eaf-8f5b-ee8ceba38aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.816094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4ea6c-efd6-4eaf-8f5b-ee8ceba38aea admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:14.817014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4ea6c-efd6-4eaf-8f5b-ee8ceba38aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:14.819173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4ea6c-efd6-4eaf-8f5b-ee8ceba38aea admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:14.863618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 940/1893] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:14 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:14.863734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909016-aed6-48ec-bd7f-6610f8dc7502 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:14.874458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909016-aed6-48ec-bd7f-6610f8dc7502 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:14.911650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4ea6c-efd6-4eaf-8f5b-ee8ceba38aea admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:14.913617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-99a4ea6c-efd6-4eaf-8f5b-ee8ceba38aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:14.952002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fe6d3-0e95-47b5-b8f0-56829c75ece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:14.952844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fe6d3-0e95-47b5-b8f0-56829c75ece4 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.953179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fe6d3-0e95-47b5-b8f0-56829c75ece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.953468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fe6d3-0e95-47b5-b8f0-56829c75ece4 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:14.954169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fe6d3-0e95-47b5-b8f0-56829c75ece4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:14.956969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fe6d3-0e95-47b5-b8f0-56829c75ece4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:14.981622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 954/1894] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:14 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:14.985773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fe6d3-0e95-47b5-b8f0-56829c75ece4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:14.990897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4ea6c-efd6-4eaf-8f5b-ee8ceba38aea admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:14.998060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fe6d3-0e95-47b5-b8f0-56829c75ece4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:14.998196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:14.999365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:14.999714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:14.999980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] PATH_INFO: `/v3/users/3720f34965344d72996ff684b8458804` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.001276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3720f34965344d72996ff684b8458804)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.003465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.004576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59b328-ae9d-43d4-8dec-f6566561a357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:15.005512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59b328-ae9d-43d4-8dec-f6566561a357 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.006025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59b328-ae9d-43d4-8dec-f6566561a357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.006480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59b328-ae9d-43d4-8dec-f6566561a357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.007403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59b328-ae9d-43d4-8dec-f6566561a357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.009130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59b328-ae9d-43d4-8dec-f6566561a357 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.018903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:15.025750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 955/1895] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:15.027436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59b328-ae9d-43d4-8dec-f6566561a357 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:15.039706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59b328-ae9d-43d4-8dec-f6566561a357 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:15.044643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ec5d9-933a-4bdc-8e4d-9962839d3c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:15.045395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ec5d9-933a-4bdc-8e4d-9962839d3c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.045678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ec5d9-933a-4bdc-8e4d-9962839d3c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.045958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ec5d9-933a-4bdc-8e4d-9962839d3c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.046775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ec5d9-933a-4bdc-8e4d-9962839d3c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.048632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ec5d9-933a-4bdc-8e4d-9962839d3c8a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.063719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 956/1896] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:15.064901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ec5d9-933a-4bdc-8e4d-9962839d3c8a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:15.070531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3720f34965344d72996ff684b8458804'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:15.074751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ec5d9-933a-4bdc-8e4d-9962839d3c8a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:15.081341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cea0a9-6024-4cd1-a7b5-f1aaf31597af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:15.082387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cea0a9-6024-4cd1-a7b5-f1aaf31597af admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.082955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cea0a9-6024-4cd1-a7b5-f1aaf31597af admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.083499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cea0a9-6024-4cd1-a7b5-f1aaf31597af admin admin] PATH_INFO: `/v3/projects/828e02fe01fc4c43b1fd33fd076157d5/users/34e402c2c710488d9ab26496f793eabc/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.083862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3720f34965344d72996ff684b8458804'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:15.093854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3720f34965344d72996ff684b8458804'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:15.102716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cea0a9-6024-4cd1-a7b5-f1aaf31597af admin admin] RBAC: Authorizing `identity:create_grant(project_id=828e02fe01fc4c43b1fd33fd076157d5, user_id=34e402c2c710488d9ab26496f793eabc, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.104572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3720f34965344d72996ff684b8458804'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:15.105441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-636713af-4e33-4fed-9b97-b548c910baaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3720f34965344d72996ff684b8458804'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:15.113517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 941/1897] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:14 2022] DELETE /identity/v3/users/3720f34965344d72996ff684b8458804 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:15.114479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cea0a9-6024-4cd1-a7b5-f1aaf31597af admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.117574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a874e-0c59-42d2-b475-02d9d740c316 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.117862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a874e-0c59-42d2-b475-02d9d740c316 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.118138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a874e-0c59-42d2-b475-02d9d740c316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.154115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 957/1898] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:15 2022] PUT /identity/v3/projects/828e02fe01fc4c43b1fd33fd076157d5/users/34e402c2c710488d9ab26496f793eabc/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:15.162925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567152ff-bc08-4a51-859f-0991e83036af None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.164008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567152ff-bc08-4a51-859f-0991e83036af None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.164615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567152ff-bc08-4a51-859f-0991e83036af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.195726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6b2a874e-0c59-42d2-b475-02d9d740c316 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:15.196406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6b2a874e-0c59-42d2-b475-02d9d740c316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:15.204839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-567152ff-bc08-4a51-859f-0991e83036af None None] MFA Rules not processed for user `34e402c2c710488d9ab26496f793eabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:15.206107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-567152ff-bc08-4a51-859f-0991e83036af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:15.299349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 942/1899] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:15.310491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a874e-0c59-42d2-b475-02d9d740c316 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:15.313408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6b2a874e-0c59-42d2-b475-02d9d740c316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:15.416885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 958/1900] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:15.422017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567152ff-bc08-4a51-859f-0991e83036af None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:15.424444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-567152ff-bc08-4a51-859f-0991e83036af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:15.440792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a874e-0c59-42d2-b475-02d9d740c316 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:15.450347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853910b1-478c-4518-bf13-c92ca12f3059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:15.450490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567152ff-bc08-4a51-859f-0991e83036af None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:15.452452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853910b1-478c-4518-bf13-c92ca12f3059 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.454200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853910b1-478c-4518-bf13-c92ca12f3059 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.454891 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853910b1-478c-4518-bf13-c92ca12f3059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.465814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b288141-b2c3-4499-b237-9e717af65987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:15.466729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b288141-b2c3-4499-b237-9e717af65987 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.467022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b288141-b2c3-4499-b237-9e717af65987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.467300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b288141-b2c3-4499-b237-9e717af65987 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.468616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b288141-b2c3-4499-b237-9e717af65987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.471071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b288141-b2c3-4499-b237-9e717af65987 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.482569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853910b1-478c-4518-bf13-c92ca12f3059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.484197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853910b1-478c-4518-bf13-c92ca12f3059 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.491510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:15.500634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 943/1901] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:36:15.516223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 959/1902] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:15 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:15.519718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853910b1-478c-4518-bf13-c92ca12f3059 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:15.532053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853910b1-478c-4518-bf13-c92ca12f3059 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:15.574795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:15.633227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61c32b-1c1b-4902-a516-b91d5d5d4f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:15.634090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61c32b-1c1b-4902-a516-b91d5d5d4f35 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.634301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61c32b-1c1b-4902-a516-b91d5d5d4f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.634578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61c32b-1c1b-4902-a516-b91d5d5d4f35 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.635495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61c32b-1c1b-4902-a516-b91d5d5d4f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.639842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61c32b-1c1b-4902-a516-b91d5d5d4f35 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.669720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 944/1903] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:15 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:36:15.674551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b288141-b2c3-4499-b237-9e717af65987 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:15.688480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b288141-b2c3-4499-b237-9e717af65987 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:15.694023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7e754-c884-4c73-98de-bf876a033d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:15.695451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7e754-c884-4c73-98de-bf876a033d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.695735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7e754-c884-4c73-98de-bf876a033d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.696007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7e754-c884-4c73-98de-bf876a033d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.696771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7e754-c884-4c73-98de-bf876a033d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.701566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7e754-c884-4c73-98de-bf876a033d1d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.737629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 960/1904] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:15.741801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61c32b-1c1b-4902-a516-b91d5d5d4f35 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:15.768725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61c32b-1c1b-4902-a516-b91d5d5d4f35 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:15.785523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabda24-e27e-4439-9b4b-b11806e0582f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:15.786231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabda24-e27e-4439-9b4b-b11806e0582f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.786513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabda24-e27e-4439-9b4b-b11806e0582f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.786797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabda24-e27e-4439-9b4b-b11806e0582f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.787712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabda24-e27e-4439-9b4b-b11806e0582f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.789255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabda24-e27e-4439-9b4b-b11806e0582f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.812942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 945/1905] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:15.814085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7e754-c884-4c73-98de-bf876a033d1d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:15.824532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7e754-c884-4c73-98de-bf876a033d1d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:15.843765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead30845-1e2b-4b6a-b440-8869c90875fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:15.845353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead30845-1e2b-4b6a-b440-8869c90875fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.845901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead30845-1e2b-4b6a-b440-8869c90875fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.846470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead30845-1e2b-4b6a-b440-8869c90875fa admin admin] PATH_INFO: `/v3/projects/5453a671afab49988f9fc38fb7a77a46/users/cf00bfc5ee3b4f6db45058253890ffa4/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.866237 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead30845-1e2b-4b6a-b440-8869c90875fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5453a671afab49988f9fc38fb7a77a46, user_id=cf00bfc5ee3b4f6db45058253890ffa4, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:15.869650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead30845-1e2b-4b6a-b440-8869c90875fa admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:15.903055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba94e05-7354-4688-bec1-bd4a842fddad None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:15.903055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba94e05-7354-4688-bec1-bd4a842fddad None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:15.903346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba94e05-7354-4688-bec1-bd4a842fddad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:15.904573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 961/1906] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:15 2022] PUT /identity/v3/projects/5453a671afab49988f9fc38fb7a77a46/users/cf00bfc5ee3b4f6db45058253890ffa4/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:15.951384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4ba94e05-7354-4688-bec1-bd4a842fddad None None] MFA Rules not processed for user `cf00bfc5ee3b4f6db45058253890ffa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:15.952645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4ba94e05-7354-4688-bec1-bd4a842fddad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:16.099067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 946/1907] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:16.103455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead30845-1e2b-4b6a-b440-8869c90875fa admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:16.225628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead30845-1e2b-4b6a-b440-8869c90875fa admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:16.235820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affcb00-5b9c-405a-a084-b0f8fb818d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:16.236565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affcb00-5b9c-405a-a084-b0f8fb818d89 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.236990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affcb00-5b9c-405a-a084-b0f8fb818d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.237319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affcb00-5b9c-405a-a084-b0f8fb818d89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:16.238199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affcb00-5b9c-405a-a084-b0f8fb818d89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:16.240519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affcb00-5b9c-405a-a084-b0f8fb818d89 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:16.287990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 962/1908] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:16 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:16.293066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba94e05-7354-4688-bec1-bd4a842fddad None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:16.305893 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba94e05-7354-4688-bec1-bd4a842fddad None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:16.310228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affcb00-5b9c-405a-a084-b0f8fb818d89 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:16.311574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0affcb00-5b9c-405a-a084-b0f8fb818d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:16.395452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affcb00-5b9c-405a-a084-b0f8fb818d89 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:16.401879 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:16.402076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66e474-95eb-443d-8b59-2d1d6f7e3a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:16.402750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66e474-95eb-443d-8b59-2d1d6f7e3a98 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.403031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66e474-95eb-443d-8b59-2d1d6f7e3a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.403124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.403302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66e474-95eb-443d-8b59-2d1d6f7e3a98 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:16.403609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.403980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] PATH_INFO: `/v3/projects/a5b92753e536433794bba24ae2e7c225` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:16.404081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66e474-95eb-443d-8b59-2d1d6f7e3a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:16.405758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b92753e536433794bba24ae2e7c225)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:16.406182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66e474-95eb-443d-8b59-2d1d6f7e3a98 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:16.408563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:16.430921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 947/1909] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:16 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:16.434954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66e474-95eb-443d-8b59-2d1d6f7e3a98 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:16.448221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66e474-95eb-443d-8b59-2d1d6f7e3a98 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:16.458225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61cd0e-7bdd-4a5f-bb7f-515cb04993fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:16.458225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61cd0e-7bdd-4a5f-bb7f-515cb04993fb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.458225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61cd0e-7bdd-4a5f-bb7f-515cb04993fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.458225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61cd0e-7bdd-4a5f-bb7f-515cb04993fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:16.458592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61cd0e-7bdd-4a5f-bb7f-515cb04993fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:16.458592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61cd0e-7bdd-4a5f-bb7f-515cb04993fb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:16.476242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 948/1910] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:16.477340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61cd0e-7bdd-4a5f-bb7f-515cb04993fb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:16.481596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=3,keystone:SELECT=94,keystone:DELETE=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:16.484507 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b92753e536433794bba24ae2e7c225', 'request_id': 'req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.488693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61cd0e-7bdd-4a5f-bb7f-515cb04993fb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:16.494251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b92753e536433794bba24ae2e7c225', 'request_id': 'req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.500216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b92753e536433794bba24ae2e7c225', 'request_id': 'req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.507138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:16.512492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] The token cache is being invalidate because project a5b92753e536433794bba24ae2e7c225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:16.512950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.513715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.515582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 963/1911] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:16 2022] DELETE /identity/v3/projects/a5b92753e536433794bba24ae2e7c225 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:16.519356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:16.520728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:16.549059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257e10-8d91-495f-99f2-e45f51fde661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:16.550137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257e10-8d91-495f-99f2-e45f51fde661 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.550429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257e10-8d91-495f-99f2-e45f51fde661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.550707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257e10-8d91-495f-99f2-e45f51fde661 admin admin] PATH_INFO: `/v3/projects/b484c127324b4b6781258c76f1dc19e6/users/6ec63a56873d4abe8cf9e4f5b4717b49/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:16.566215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db257e10-8d91-495f-99f2-e45f51fde661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b484c127324b4b6781258c76f1dc19e6, user_id=6ec63a56873d4abe8cf9e4f5b4717b49, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:16.569783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db257e10-8d91-495f-99f2-e45f51fde661 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:16.591961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 949/1912] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:16 2022] PUT /identity/v3/projects/b484c127324b4b6781258c76f1dc19e6/users/6ec63a56873d4abe8cf9e4f5b4717b49/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:16.592098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5fb1f-e0d6-4da9-9a72-d94a9379c634 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:16.593278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257e10-8d91-495f-99f2-e45f51fde661 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:16.599411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-db257e10-8d91-495f-99f2-e45f51fde661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:16.667386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:16.668292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.668578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.668885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] PATH_INFO: `/v3/users/7eea1231baa2408a9955a1dbab366d4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:16.670173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eea1231baa2408a9955a1dbab366d4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:16.672196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:16.688361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257e10-8d91-495f-99f2-e45f51fde661 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:16.693754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6478c1-e9d3-4cdc-b07e-af8b026c8a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:16.694443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6478c1-e9d3-4cdc-b07e-af8b026c8a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.694751 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6478c1-e9d3-4cdc-b07e-af8b026c8a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.695029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6478c1-e9d3-4cdc-b07e-af8b026c8a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:16.695781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6478c1-e9d3-4cdc-b07e-af8b026c8a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:16.697436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6478c1-e9d3-4cdc-b07e-af8b026c8a79 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:16.708986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 950/1913] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:16.713624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6478c1-e9d3-4cdc-b07e-af8b026c8a79 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:16.724832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6478c1-e9d3-4cdc-b07e-af8b026c8a79 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:16.733196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d29f3-9ca4-4edc-a5f6-c242e22ef8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:16.734235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d29f3-9ca4-4edc-a5f6-c242e22ef8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.734525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d29f3-9ca4-4edc-a5f6-c242e22ef8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.734923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d29f3-9ca4-4edc-a5f6-c242e22ef8bb admin admin] PATH_INFO: `/v3/projects/b484c127324b4b6781258c76f1dc19e6/users/6ec63a56873d4abe8cf9e4f5b4717b49/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:16.735186 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eea1231baa2408a9955a1dbab366d4e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.737900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d29f3-9ca4-4edc-a5f6-c242e22ef8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b484c127324b4b6781258c76f1dc19e6, user_id=6ec63a56873d4abe8cf9e4f5b4717b49, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:16.740830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d29f3-9ca4-4edc-a5f6-c242e22ef8bb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:16.750306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eea1231baa2408a9955a1dbab366d4e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.756782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 951/1914] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:16 2022] PUT /identity/v3/projects/b484c127324b4b6781258c76f1dc19e6/users/6ec63a56873d4abe8cf9e4f5b4717b49/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:16.763988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbe88c-afc5-4f39-9677-4bafddedec23 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.764235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eea1231baa2408a9955a1dbab366d4e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.764472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbe88c-afc5-4f39-9677-4bafddedec23 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.764699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbe88c-afc5-4f39-9677-4bafddedec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:16.764934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eea1231baa2408a9955a1dbab366d4e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.773235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eea1231baa2408a9955a1dbab366d4e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:16.775525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:16.783257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 964/1915] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:16 2022] DELETE /identity/v3/users/7eea1231baa2408a9955a1dbab366d4e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:16.801886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c8fbe88c-afc5-4f39-9677-4bafddedec23 None None] MFA Rules not processed for user `6ec63a56873d4abe8cf9e4f5b4717b49`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:16.803496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c8fbe88c-afc5-4f39-9677-4bafddedec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:16.904436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 952/1916] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:16.917311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:16.918769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:16.986790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1339e-b633-4e23-8f29-e22498cba7a6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:16.990949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0bb20-0b6d-4969-b119-9ffa70aeb40f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:16.991684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0bb20-0b6d-4969-b119-9ffa70aeb40f service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:16.992150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0bb20-0b6d-4969-b119-9ffa70aeb40f service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:16.992467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0bb20-0b6d-4969-b119-9ffa70aeb40f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.005256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0bb20-0b6d-4969-b119-9ffa70aeb40f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.007665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0bb20-0b6d-4969-b119-9ffa70aeb40f service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:17.018938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 965/1917] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:17.068623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbe88c-afc5-4f39-9677-4bafddedec23 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:17.079993 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbe88c-afc5-4f39-9677-4bafddedec23 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:17.083970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda432bc-f15c-4947-b415-eb29668bf900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:17.084753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda432bc-f15c-4947-b415-eb29668bf900 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.085139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda432bc-f15c-4947-b415-eb29668bf900 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.085489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda432bc-f15c-4947-b415-eb29668bf900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.088580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cda432bc-f15c-4947-b415-eb29668bf900 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:17.090402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0bb20-0b6d-4969-b119-9ffa70aeb40f service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:17.092854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a6e0bb20-0b6d-4969-b119-9ffa70aeb40f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:17.113569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:17.187928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0bb20-0b6d-4969-b119-9ffa70aeb40f service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:17.194894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:17.195851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.196134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.196420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] PATH_INFO: `/v3/projects/92f9a0d4561548f9becad44d0fca601b` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.197779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f9a0d4561548f9becad44d0fca601b)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.199844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:17.219234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda432bc-f15c-4947-b415-eb29668bf900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.221466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda432bc-f15c-4947-b415-eb29668bf900 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:17.238940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 953/1918] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:17.283087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f9a0d4561548f9becad44d0fca601b', 'request_id': 'req-697a55cc-ac97-476b-8daf-eb71633d9b2f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:17.298617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f9a0d4561548f9becad44d0fca601b', 'request_id': 'req-697a55cc-ac97-476b-8daf-eb71633d9b2f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:17.302979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f9a0d4561548f9becad44d0fca601b', 'request_id': 'req-697a55cc-ac97-476b-8daf-eb71633d9b2f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:17.317512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] The token cache is being invalidate because project 92f9a0d4561548f9becad44d0fca601b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:17.317834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:17.318694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-697a55cc-ac97-476b-8daf-eb71633d9b2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:17.322082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 966/1919] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:17 2022] DELETE /identity/v3/projects/92f9a0d4561548f9becad44d0fca601b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:17.327906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b2bca-6a90-425e-8adf-da0246adb3cf None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.328380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b2bca-6a90-425e-8adf-da0246adb3cf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.328726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b2bca-6a90-425e-8adf-da0246adb3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.349694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cc0b2bca-6a90-425e-8adf-da0246adb3cf None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:17.351009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cc0b2bca-6a90-425e-8adf-da0246adb3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:17.436601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 954/1920] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:17.444968 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da54cb-72bc-4e6e-bf95-49824e3fe93d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.445181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da54cb-72bc-4e6e-bf95-49824e3fe93d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.445492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da54cb-72bc-4e6e-bf95-49824e3fe93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.469757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-63da54cb-72bc-4e6e-bf95-49824e3fe93d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:17.471422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-63da54cb-72bc-4e6e-bf95-49824e3fe93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:17.480080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 967/1921] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:17.484390 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b2bca-6a90-425e-8adf-da0246adb3cf None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:17.494913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b2bca-6a90-425e-8adf-da0246adb3cf None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:17.500155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df3276-4220-4d62-a63b-7d62084686ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:17.500872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df3276-4220-4d62-a63b-7d62084686ee admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.501259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df3276-4220-4d62-a63b-7d62084686ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.501661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df3276-4220-4d62-a63b-7d62084686ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.502704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df3276-4220-4d62-a63b-7d62084686ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:17.503082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df3276-4220-4d62-a63b-7d62084686ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.504805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df3276-4220-4d62-a63b-7d62084686ee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:17.519471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 955/1922] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:17.523710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da54cb-72bc-4e6e-bf95-49824e3fe93d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:17.534465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da54cb-72bc-4e6e-bf95-49824e3fe93d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:17.539412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4afe-b294-43cb-a4c3-b015fe2c6e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:17.540082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4afe-b294-43cb-a4c3-b015fe2c6e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.540366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4afe-b294-43cb-a4c3-b015fe2c6e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.540648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4afe-b294-43cb-a4c3-b015fe2c6e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.541401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b4afe-b294-43cb-a4c3-b015fe2c6e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.543316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b4afe-b294-43cb-a4c3-b015fe2c6e0e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:17.574315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 968/1923] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:17 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:17.575656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df3276-4220-4d62-a63b-7d62084686ee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:17.586649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df3276-4220-4d62-a63b-7d62084686ee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:17.647642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746b885-c849-46ed-86af-0ad50b2b380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:17.648404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746b885-c849-46ed-86af-0ad50b2b380f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.648828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746b885-c849-46ed-86af-0ad50b2b380f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.649121 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746b885-c849-46ed-86af-0ad50b2b380f admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.649897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746b885-c849-46ed-86af-0ad50b2b380f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.651886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746b885-c849-46ed-86af-0ad50b2b380f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:17.680120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 956/1924] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:17 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:17.681438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4afe-b294-43cb-a4c3-b015fe2c6e0e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:17.688327 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:17.692795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4afe-b294-43cb-a4c3-b015fe2c6e0e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:17.698726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa30e0-97cf-4e72-a332-c722ad260c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:17.699448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa30e0-97cf-4e72-a332-c722ad260c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.699764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa30e0-97cf-4e72-a332-c722ad260c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.700221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa30e0-97cf-4e72-a332-c722ad260c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.701184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa30e0-97cf-4e72-a332-c722ad260c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.702984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa30e0-97cf-4e72-a332-c722ad260c9e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:17.715445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 969/1925] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:17.719538 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746b885-c849-46ed-86af-0ad50b2b380f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:17.729633 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746b885-c849-46ed-86af-0ad50b2b380f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:17.734035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51acfe-c5d0-4286-b2b5-7d36f771b1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:17.734692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51acfe-c5d0-4286-b2b5-7d36f771b1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.734970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51acfe-c5d0-4286-b2b5-7d36f771b1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.735244 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51acfe-c5d0-4286-b2b5-7d36f771b1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.736209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51acfe-c5d0-4286-b2b5-7d36f771b1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.737873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51acfe-c5d0-4286-b2b5-7d36f771b1a7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:17.748379 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 957/1926] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:17.753840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa30e0-97cf-4e72-a332-c722ad260c9e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:17.763873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa30e0-97cf-4e72-a332-c722ad260c9e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:17.768702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961879a-ce37-4639-bb47-d6ab16a90bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:17.769616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961879a-ce37-4639-bb47-d6ab16a90bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.769899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961879a-ce37-4639-bb47-d6ab16a90bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.770179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961879a-ce37-4639-bb47-d6ab16a90bb3 admin admin] PATH_INFO: `/v3/projects/26e5ab32bfa0441ea5bb5a3b53c36dc9/users/4801e63227af4a8baf6932c08bc4acaf/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.785451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961879a-ce37-4639-bb47-d6ab16a90bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e5ab32bfa0441ea5bb5a3b53c36dc9, user_id=4801e63227af4a8baf6932c08bc4acaf, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.788144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961879a-ce37-4639-bb47-d6ab16a90bb3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:17.803910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 970/1927] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:17 2022] PUT /identity/v3/projects/26e5ab32bfa0441ea5bb5a3b53c36dc9/users/4801e63227af4a8baf6932c08bc4acaf/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:17.815649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ec38-b93a-4bae-bb46-24ffa3feacd3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.815933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ec38-b93a-4bae-bb46-24ffa3feacd3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.816203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ec38-b93a-4bae-bb46-24ffa3feacd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.845051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a2a1ec38-b93a-4bae-bb46-24ffa3feacd3 None None] MFA Rules not processed for user `4801e63227af4a8baf6932c08bc4acaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:17.846330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a2a1ec38-b93a-4bae-bb46-24ffa3feacd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:17.850616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:17.914511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 958/1928] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:17.919112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961879a-ce37-4639-bb47-d6ab16a90bb3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:17.987346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961879a-ce37-4639-bb47-d6ab16a90bb3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:17.992347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a3ff3-9165-49a4-b306-3cea545bbf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:17.993122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a3ff3-9165-49a4-b306-3cea545bbf13 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:17.993407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a3ff3-9165-49a4-b306-3cea545bbf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:17.993685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a3ff3-9165-49a4-b306-3cea545bbf13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:17.994384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a3ff3-9165-49a4-b306-3cea545bbf13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:17.996382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a3ff3-9165-49a4-b306-3cea545bbf13 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:18.023653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 971/1929] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:17 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:18.028112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1ec38-b93a-4bae-bb46-24ffa3feacd3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:18.039945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1ec38-b93a-4bae-bb46-24ffa3feacd3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:18.097954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4345d85-0ca6-4212-a6d1-c3170abb898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:18.098644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4345d85-0ca6-4212-a6d1-c3170abb898c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:18.098925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4345d85-0ca6-4212-a6d1-c3170abb898c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:18.099199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4345d85-0ca6-4212-a6d1-c3170abb898c admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:18.099903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4345d85-0ca6-4212-a6d1-c3170abb898c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:18.101925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4345d85-0ca6-4212-a6d1-c3170abb898c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:18.124516 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 959/1930] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:18 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:18.128629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a3ff3-9165-49a4-b306-3cea545bbf13 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:18.139744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a3ff3-9165-49a4-b306-3cea545bbf13 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:18.145383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57699fe0-e93d-41a9-a3cc-b821a48ce349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:18.146048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57699fe0-e93d-41a9-a3cc-b821a48ce349 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:18.146330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57699fe0-e93d-41a9-a3cc-b821a48ce349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:18.146604 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57699fe0-e93d-41a9-a3cc-b821a48ce349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:18.147359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57699fe0-e93d-41a9-a3cc-b821a48ce349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:18.148990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57699fe0-e93d-41a9-a3cc-b821a48ce349 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:18.160189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 972/1931] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:18.164462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4345d85-0ca6-4212-a6d1-c3170abb898c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:18.174894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4345d85-0ca6-4212-a6d1-c3170abb898c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:18.179799 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac384794-35bc-43e4-9505-06bf9e84b935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:18.180694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac384794-35bc-43e4-9505-06bf9e84b935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:18.181003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac384794-35bc-43e4-9505-06bf9e84b935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:18.181280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac384794-35bc-43e4-9505-06bf9e84b935 admin admin] PATH_INFO: `/v3/projects/a3d4ea095ed04e7b9960b10cb85e9fa4/users/6b3aae266d90400bba868e86b276f5dc/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:18.196198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac384794-35bc-43e4-9505-06bf9e84b935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d4ea095ed04e7b9960b10cb85e9fa4, user_id=6b3aae266d90400bba868e86b276f5dc, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:18.198887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac384794-35bc-43e4-9505-06bf9e84b935 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:18.214190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 960/1932] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:18 2022] PUT /identity/v3/projects/a3d4ea095ed04e7b9960b10cb85e9fa4/users/6b3aae266d90400bba868e86b276f5dc/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:18.220153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57699fe0-e93d-41a9-a3cc-b821a48ce349 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:18.230585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57699fe0-e93d-41a9-a3cc-b821a48ce349 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:18.292174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142765a2-3ddd-4b90-bd92-f256c5aaa5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:18.292876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142765a2-3ddd-4b90-bd92-f256c5aaa5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:18.293160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142765a2-3ddd-4b90-bd92-f256c5aaa5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:18.293438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142765a2-3ddd-4b90-bd92-f256c5aaa5e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:18.294156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142765a2-3ddd-4b90-bd92-f256c5aaa5e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:18.295759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142765a2-3ddd-4b90-bd92-f256c5aaa5e5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:18.306146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 973/1933] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:18.310503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac384794-35bc-43e4-9505-06bf9e84b935 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:18.321103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac384794-35bc-43e4-9505-06bf9e84b935 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:18.326169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5db8f-bea7-40cf-ae28-37dbe29f0bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:18.327060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5db8f-bea7-40cf-ae28-37dbe29f0bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:18.327343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5db8f-bea7-40cf-ae28-37dbe29f0bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:18.327620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5db8f-bea7-40cf-ae28-37dbe29f0bc9 admin admin] PATH_INFO: `/v3/projects/a3d4ea095ed04e7b9960b10cb85e9fa4/users/6b3aae266d90400bba868e86b276f5dc/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:18.329783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5db8f-bea7-40cf-ae28-37dbe29f0bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d4ea095ed04e7b9960b10cb85e9fa4, user_id=6b3aae266d90400bba868e86b276f5dc, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:18.332614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5db8f-bea7-40cf-ae28-37dbe29f0bc9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:18.345258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 961/1934] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:18 2022] PUT /identity/v3/projects/a3d4ea095ed04e7b9960b10cb85e9fa4/users/6b3aae266d90400bba868e86b276f5dc/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:18.356912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4158f3-42c3-4995-9ed6-b182a658a43f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:18.357315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4158f3-42c3-4995-9ed6-b182a658a43f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:18.357732 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4158f3-42c3-4995-9ed6-b182a658a43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:18.389118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5a4158f3-42c3-4995-9ed6-b182a658a43f None None] MFA Rules not processed for user `6b3aae266d90400bba868e86b276f5dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:18.390318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5a4158f3-42c3-4995-9ed6-b182a658a43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:18.425430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:18.474301 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 974/1935] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:18.486131 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5db8f-bea7-40cf-ae28-37dbe29f0bc9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:18.487978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c4a5db8f-bea7-40cf-ae28-37dbe29f0bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:18.564840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5db8f-bea7-40cf-ae28-37dbe29f0bc9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:18.569261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232b480-d662-48b3-8c16-fbfba98627b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:18.570175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232b480-d662-48b3-8c16-fbfba98627b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:18.570947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232b480-d662-48b3-8c16-fbfba98627b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:18.572009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232b480-d662-48b3-8c16-fbfba98627b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:18.584520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232b480-d662-48b3-8c16-fbfba98627b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:18.586249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232b480-d662-48b3-8c16-fbfba98627b1 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:18.592380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:18.598910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 962/1936] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:18.755243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4158f3-42c3-4995-9ed6-b182a658a43f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:18.833614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4158f3-42c3-4995-9ed6-b182a658a43f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:18.838231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:18.839447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:18.840054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:18.840335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] PATH_INFO: `/v3/users/4801e63227af4a8baf6932c08bc4acaf` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:18.842095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4801e63227af4a8baf6932c08bc4acaf)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:18.844644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:18.905727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4801e63227af4a8baf6932c08bc4acaf'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:18.917674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4801e63227af4a8baf6932c08bc4acaf'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:18.927762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4801e63227af4a8baf6932c08bc4acaf'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:18.928444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4801e63227af4a8baf6932c08bc4acaf'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:18.944810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4801e63227af4a8baf6932c08bc4acaf'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:18.955455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 975/1937] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:18 2022] DELETE /identity/v3/users/4801e63227af4a8baf6932c08bc4acaf => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:18.960395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc31c94-275d-4b22-a5c6-6cf24dc02611 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:18.960683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc31c94-275d-4b22-a5c6-6cf24dc02611 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:18.961002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc31c94-275d-4b22-a5c6-6cf24dc02611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:18.984626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9cc31c94-275d-4b22-a5c6-6cf24dc02611 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:18.986374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9cc31c94-275d-4b22-a5c6-6cf24dc02611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:19.076705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 963/1938] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:19.087823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:19.089232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:19.145676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:19.158433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f413a88-c059-466d-a8e7-60a8c8645a9e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:19.163050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569caa7-bf3c-4ea6-a3c1-241ee37a7908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:19.164009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569caa7-bf3c-4ea6-a3c1-241ee37a7908 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:19.164612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569caa7-bf3c-4ea6-a3c1-241ee37a7908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:19.165000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569caa7-bf3c-4ea6-a3c1-241ee37a7908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:19.182559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569caa7-bf3c-4ea6-a3c1-241ee37a7908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:19.184023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569caa7-bf3c-4ea6-a3c1-241ee37a7908 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:19.198287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 976/1939] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:19.479091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc31c94-275d-4b22-a5c6-6cf24dc02611 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:19.480917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9cc31c94-275d-4b22-a5c6-6cf24dc02611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:19.499604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc31c94-275d-4b22-a5c6-6cf24dc02611 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:19.506461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:19.507424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:19.507711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:19.508041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] PATH_INFO: `/v3/users/363acb6f119a40eea2cf174fd9e09598` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:19.509582 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=363acb6f119a40eea2cf174fd9e09598)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:19.512574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:19.599365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '363acb6f119a40eea2cf174fd9e09598'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.615221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '363acb6f119a40eea2cf174fd9e09598'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.625340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '363acb6f119a40eea2cf174fd9e09598'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.641339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569caa7-bf3c-4ea6-a3c1-241ee37a7908 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:19.643597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9569caa7-bf3c-4ea6-a3c1-241ee37a7908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:19.655178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '363acb6f119a40eea2cf174fd9e09598'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.658905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66cd94e5-46d2-4d57-942c-ee03f3e1d384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '363acb6f119a40eea2cf174fd9e09598'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.662761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569caa7-bf3c-4ea6-a3c1-241ee37a7908 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:19.678182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:19.679399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:19.680547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:19.680950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] PATH_INFO: `/v3/projects/26e5ab32bfa0441ea5bb5a3b53c36dc9` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:19.686757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 964/1940] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:19 2022] DELETE /identity/v3/users/363acb6f119a40eea2cf174fd9e09598 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:19.689969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e5ab32bfa0441ea5bb5a3b53c36dc9)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:19.691636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9ed8d-864d-4b44-80b0-bbee853fa0af None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:19.692085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9ed8d-864d-4b44-80b0-bbee853fa0af None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:19.692196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:19.692588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9ed8d-864d-4b44-80b0-bbee853fa0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:19.754987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f9f9ed8d-864d-4b44-80b0-bbee853fa0af None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:19.757980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f9f9ed8d-864d-4b44-80b0-bbee853fa0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:19.809825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e5ab32bfa0441ea5bb5a3b53c36dc9', 'request_id': 'req-441fb555-261a-4d76-9847-7090eec96653'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.830576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e5ab32bfa0441ea5bb5a3b53c36dc9', 'request_id': 'req-441fb555-261a-4d76-9847-7090eec96653'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.840923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e5ab32bfa0441ea5bb5a3b53c36dc9', 'request_id': 'req-441fb555-261a-4d76-9847-7090eec96653'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.869823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] The token cache is being invalidate because project 26e5ab32bfa0441ea5bb5a3b53c36dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:19.869823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.873613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 965/1941] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:19.889586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9ed8d-864d-4b44-80b0-bbee853fa0af None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:19.897265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f9f9ed8d-864d-4b44-80b0-bbee853fa0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:19.907281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:19.911992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 977/1942] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:19 2022] DELETE /identity/v3/projects/26e5ab32bfa0441ea5bb5a3b53c36dc9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:19.918680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:19.919909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:19.921642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:19.947127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fb555-261a-4d76-9847-7090eec96653 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:19.951981 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:19.953105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:19.953389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:19.953666 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] PATH_INFO: `/v3/users/6b3aae266d90400bba868e86b276f5dc` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:19.955185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b3aae266d90400bba868e86b276f5dc)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:19.958388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:20.033531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9ed8d-864d-4b44-80b0-bbee853fa0af None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:20.048063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855455d4-4602-4617-9b77-ea170b794e11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:20.048782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855455d4-4602-4617-9b77-ea170b794e11 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:20.049247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855455d4-4602-4617-9b77-ea170b794e11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:20.049510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855455d4-4602-4617-9b77-ea170b794e11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:20.051934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-855455d4-4602-4617-9b77-ea170b794e11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:20.065258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b3aae266d90400bba868e86b276f5dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:20.090740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855455d4-4602-4617-9b77-ea170b794e11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:20.096480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855455d4-4602-4617-9b77-ea170b794e11 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:20.097712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b3aae266d90400bba868e86b276f5dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:20.109031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b3aae266d90400bba868e86b276f5dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:20.109726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b3aae266d90400bba868e86b276f5dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:20.118592 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b3aae266d90400bba868e86b276f5dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:20.124889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 966/1943] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:20.126125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 978/1944] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:19 2022] DELETE /identity/v3/users/6b3aae266d90400bba868e86b276f5dc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:20.892272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855455d4-4602-4617-9b77-ea170b794e11 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:20.909302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855455d4-4602-4617-9b77-ea170b794e11 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:20.987691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:20.989314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:21.003758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.004773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.005115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.005525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] PATH_INFO: `/v3/projects/a3d4ea095ed04e7b9960b10cb85e9fa4` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.006836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d4ea095ed04e7b9960b10cb85e9fa4)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.008807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.043773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:21.073927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5cd75-0b7c-4b2d-a2f2-6be78b84f0cd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.087773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d4ea095ed04e7b9960b10cb85e9fa4', 'request_id': 'req-83dd2d04-ad56-4c99-bc16-5be0da87915e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.098950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d4ea095ed04e7b9960b10cb85e9fa4', 'request_id': 'req-83dd2d04-ad56-4c99-bc16-5be0da87915e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.114177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d4ea095ed04e7b9960b10cb85e9fa4', 'request_id': 'req-83dd2d04-ad56-4c99-bc16-5be0da87915e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.152155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] The token cache is being invalidate because project a3d4ea095ed04e7b9960b10cb85e9fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:21.161642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.162531 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.162816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.163093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] PATH_INFO: `/v3/projects/4139f18358794e7b8ea7cb2486864847` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.165898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4139f18358794e7b8ea7cb2486864847)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.166174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.167057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-83dd2d04-ad56-4c99-bc16-5be0da87915e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.168073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.172034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 967/1945] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:20 2022] DELETE /identity/v3/projects/a3d4ea095ed04e7b9960b10cb85e9fa4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:21.173816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea78c9-6f82-4e71-b430-aa259ad1868f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.174197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea78c9-6f82-4e71-b430-aa259ad1868f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.174480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea78c9-6f82-4e71-b430-aa259ad1868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.197640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-77ea78c9-6f82-4e71-b430-aa259ad1868f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:21.199097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-77ea78c9-6f82-4e71-b430-aa259ad1868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:21.215960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 968/1946] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:21.222784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e5c0c-c86e-45ab-93c8-96dc1f15183b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.223119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e5c0c-c86e-45ab-93c8-96dc1f15183b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.223408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e5c0c-c86e-45ab-93c8-96dc1f15183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.230386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4139f18358794e7b8ea7cb2486864847', 'request_id': 'req-f76f9cf2-b168-445e-9211-c57626367e37'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.246149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4139f18358794e7b8ea7cb2486864847', 'request_id': 'req-f76f9cf2-b168-445e-9211-c57626367e37'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.246502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f51e5c0c-c86e-45ab-93c8-96dc1f15183b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:21.247721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f51e5c0c-c86e-45ab-93c8-96dc1f15183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:21.254061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4139f18358794e7b8ea7cb2486864847', 'request_id': 'req-f76f9cf2-b168-445e-9211-c57626367e37'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.276109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] The token cache is being invalidate because project 4139f18358794e7b8ea7cb2486864847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:21.276622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.280031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f76f9cf2-b168-445e-9211-c57626367e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.282448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 979/1947] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:20 2022] DELETE /identity/v3/projects/4139f18358794e7b8ea7cb2486864847 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:21.289552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59f217-2a09-4472-9919-c892172c315c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.290020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59f217-2a09-4472-9919-c892172c315c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.290346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59f217-2a09-4472-9919-c892172c315c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.310792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:21.320862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ab59f217-2a09-4472-9919-c892172c315c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:21.322418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ab59f217-2a09-4472-9919-c892172c315c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:21.344212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 969/1948] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:21.348634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e5c0c-c86e-45ab-93c8-96dc1f15183b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.358231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e5c0c-c86e-45ab-93c8-96dc1f15183b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.364261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e94fe-2b59-43ed-acba-89c5184dc761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.364631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e94fe-2b59-43ed-acba-89c5184dc761 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.365063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e94fe-2b59-43ed-acba-89c5184dc761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.365339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e94fe-2b59-43ed-acba-89c5184dc761 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.366076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e94fe-2b59-43ed-acba-89c5184dc761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:21.366384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e94fe-2b59-43ed-acba-89c5184dc761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.368230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e94fe-2b59-43ed-acba-89c5184dc761 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.393380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 970/1949] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:36:21.395342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e94fe-2b59-43ed-acba-89c5184dc761 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.407208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 980/1950] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:21.414954 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba813f-cc00-449c-9aba-23d323ac1b17 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.415327 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba813f-cc00-449c-9aba-23d323ac1b17 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.415515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba813f-cc00-449c-9aba-23d323ac1b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.423473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e94fe-2b59-43ed-acba-89c5184dc761 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.431157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b715762-5cb6-4f01-8ff4-60583a95a8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.432608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b715762-5cb6-4f01-8ff4-60583a95a8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.433157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b715762-5cb6-4f01-8ff4-60583a95a8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.433628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b715762-5cb6-4f01-8ff4-60583a95a8b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.434887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b715762-5cb6-4f01-8ff4-60583a95a8b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.437105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b715762-5cb6-4f01-8ff4-60583a95a8b4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.437724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a5ba813f-cc00-449c-9aba-23d323ac1b17 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:21.441067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a5ba813f-cc00-449c-9aba-23d323ac1b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:21.460909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 981/1951] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:21.463623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba813f-cc00-449c-9aba-23d323ac1b17 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.479352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba813f-cc00-449c-9aba-23d323ac1b17 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.485391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54dc6e-1849-4bb9-8004-181bf05824b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.486581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54dc6e-1849-4bb9-8004-181bf05824b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.487295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54dc6e-1849-4bb9-8004-181bf05824b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.487770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54dc6e-1849-4bb9-8004-181bf05824b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.488301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 971/1952] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:21 2022] POST /identity/v3/projects => generated 409 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:21.489123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54dc6e-1849-4bb9-8004-181bf05824b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:21.489373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b715762-5cb6-4f01-8ff4-60583a95a8b4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.489971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54dc6e-1849-4bb9-8004-181bf05824b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.493201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54dc6e-1849-4bb9-8004-181bf05824b4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.507518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:21.515298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 982/1953] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:21.516878 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54dc6e-1849-4bb9-8004-181bf05824b4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.527989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54dc6e-1849-4bb9-8004-181bf05824b4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.576651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b715762-5cb6-4f01-8ff4-60583a95a8b4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.586425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9032de-dd91-4b86-8466-941490ac5769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.587128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9032de-dd91-4b86-8466-941490ac5769 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.587418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9032de-dd91-4b86-8466-941490ac5769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.587698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9032de-dd91-4b86-8466-941490ac5769 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.588402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9032de-dd91-4b86-8466-941490ac5769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.591211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9032de-dd91-4b86-8466-941490ac5769 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.591556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86395365-4721-40bc-9d78-17de26632583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.592232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86395365-4721-40bc-9d78-17de26632583 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.592515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86395365-4721-40bc-9d78-17de26632583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.592872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86395365-4721-40bc-9d78-17de26632583 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.593667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86395365-4721-40bc-9d78-17de26632583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.595823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86395365-4721-40bc-9d78-17de26632583 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.625150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 972/1954] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:21 2022] POST /identity/v3/users => generated 305 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:36:21.627318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9032de-dd91-4b86-8466-941490ac5769 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.644788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 983/1955] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:21 2022] POST /identity/v3/projects => generated 391 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:21.647672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86395365-4721-40bc-9d78-17de26632583 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.649865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9032de-dd91-4b86-8466-941490ac5769 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.730498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86395365-4721-40bc-9d78-17de26632583 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.731152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ec9cd-b78f-4e64-ba24-0d5948a8139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.731917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ec9cd-b78f-4e64-ba24-0d5948a8139b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.732268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ec9cd-b78f-4e64-ba24-0d5948a8139b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.732609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ec9cd-b78f-4e64-ba24-0d5948a8139b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.733458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ec9cd-b78f-4e64-ba24-0d5948a8139b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.735582 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ec9cd-b78f-4e64-ba24-0d5948a8139b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.737209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8648687-10ea-4d52-8c71-5221c0613b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.737983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8648687-10ea-4d52-8c71-5221c0613b69 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.738330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8648687-10ea-4d52-8c71-5221c0613b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.738664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8648687-10ea-4d52-8c71-5221c0613b69 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.739580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8648687-10ea-4d52-8c71-5221c0613b69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.742142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8648687-10ea-4d52-8c71-5221c0613b69 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.751335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 973/1956] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:21.756779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ec9cd-b78f-4e64-ba24-0d5948a8139b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.772833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ec9cd-b78f-4e64-ba24-0d5948a8139b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.778820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb00d1-4bfe-468e-ad79-d2a5f1eb9a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.779618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb00d1-4bfe-468e-ad79-d2a5f1eb9a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.779975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb00d1-4bfe-468e-ad79-d2a5f1eb9a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.780324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb00d1-4bfe-468e-ad79-d2a5f1eb9a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.781382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb00d1-4bfe-468e-ad79-d2a5f1eb9a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.783431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb00d1-4bfe-468e-ad79-d2a5f1eb9a69 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.796386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 974/1957] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:21.800528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb00d1-4bfe-468e-ad79-d2a5f1eb9a69 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.806004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 984/1958] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:21 2022] POST /identity/v3/users => generated 296 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:21.808606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8648687-10ea-4d52-8c71-5221c0613b69 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.823313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb00d1-4bfe-468e-ad79-d2a5f1eb9a69 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.827647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8648687-10ea-4d52-8c71-5221c0613b69 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.831838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38562f38-c2ca-4878-bbf1-e8bdfe3e25df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.832817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38562f38-c2ca-4878-bbf1-e8bdfe3e25df admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.833112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38562f38-c2ca-4878-bbf1-e8bdfe3e25df admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.833395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38562f38-c2ca-4878-bbf1-e8bdfe3e25df admin admin] PATH_INFO: `/v3/projects/efd6fb82be9f45fe933b71c7ae99f3ec/users/43a126f2963e46e3854595fc7de76ed9/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.838461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc6697-7bdb-4bc2-910f-ea43e99d93da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.839476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc6697-7bdb-4bc2-910f-ea43e99d93da admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.840124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc6697-7bdb-4bc2-910f-ea43e99d93da admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.840409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc6697-7bdb-4bc2-910f-ea43e99d93da admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.841195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc6697-7bdb-4bc2-910f-ea43e99d93da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.845720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc6697-7bdb-4bc2-910f-ea43e99d93da admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.857984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 985/1959] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:21.858936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc6697-7bdb-4bc2-910f-ea43e99d93da admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.862114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4abc6697-7bdb-4bc2-910f-ea43e99d93da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:21.862240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38562f38-c2ca-4878-bbf1-e8bdfe3e25df admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd6fb82be9f45fe933b71c7ae99f3ec, user_id=43a126f2963e46e3854595fc7de76ed9, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.864963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38562f38-c2ca-4878-bbf1-e8bdfe3e25df admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.868830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:21.876875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc6697-7bdb-4bc2-910f-ea43e99d93da admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.882128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.883110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.883404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.883691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] PATH_INFO: `/v3/users/34e402c2c710488d9ab26496f793eabc` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.884942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e402c2c710488d9ab26496f793eabc)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.886880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.887917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 975/1960] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:21 2022] PUT /identity/v3/projects/efd6fb82be9f45fe933b71c7ae99f3ec/users/43a126f2963e46e3854595fc7de76ed9/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:21.889136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38562f38-c2ca-4878-bbf1-e8bdfe3e25df admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:21.968479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e402c2c710488d9ab26496f793eabc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:21.975535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38562f38-c2ca-4878-bbf1-e8bdfe3e25df admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:21.985758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9afb8-18b4-4409-a2a1-93d0e6b68b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:21.987686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9afb8-18b4-4409-a2a1-93d0e6b68b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:21.988132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9afb8-18b4-4409-a2a1-93d0e6b68b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:21.988608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9afb8-18b4-4409-a2a1-93d0e6b68b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:21.989661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9afb8-18b4-4409-a2a1-93d0e6b68b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:21.992032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9afb8-18b4-4409-a2a1-93d0e6b68b1d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:21.992991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:21.994527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e402c2c710488d9ab26496f793eabc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:22.007398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e402c2c710488d9ab26496f793eabc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:22.008353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e402c2c710488d9ab26496f793eabc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:22.008832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 976/1961] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:22.010616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1682f0-afc4-4f56-a77e-45688b985fc8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.011087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1682f0-afc4-4f56-a77e-45688b985fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.011851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1682f0-afc4-4f56-a77e-45688b985fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.020558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e402c2c710488d9ab26496f793eabc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:22.028817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 986/1962] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:21 2022] DELETE /identity/v3/users/34e402c2c710488d9ab26496f793eabc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:22.033498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:22.035282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.047802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8b1682f0-afc4-4f56-a77e-45688b985fc8 None None] MFA Rules not processed for user `43a126f2963e46e3854595fc7de76ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:22.051102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8b1682f0-afc4-4f56-a77e-45688b985fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.127226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9f001-8006-4f27-9ec8-e5b043a39b0e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:22.133002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3474c3-b6da-473f-94ea-d0c3874ad521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:22.134179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3474c3-b6da-473f-94ea-d0c3874ad521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.134685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3474c3-b6da-473f-94ea-d0c3874ad521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.135732 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3474c3-b6da-473f-94ea-d0c3874ad521 admin admin] PATH_INFO: `/v3/projects/ee47f7edd88042a3a9508e3e24343c86/users/3ade449bab27445693224e6bf1e881ac/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.136322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 977/1963] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:22.138212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eaf82-cc98-4f7b-991b-79773bd7fc5d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.138666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eaf82-cc98-4f7b-991b-79773bd7fc5d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.139168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eaf82-cc98-4f7b-991b-79773bd7fc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.154952 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3474c3-b6da-473f-94ea-d0c3874ad521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee47f7edd88042a3a9508e3e24343c86, user_id=3ade449bab27445693224e6bf1e881ac, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:22.162255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-668eaf82-cc98-4f7b-991b-79773bd7fc5d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:22.164527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-668eaf82-cc98-4f7b-991b-79773bd7fc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.168770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3474c3-b6da-473f-94ea-d0c3874ad521 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:22.177096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 978/1964] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:22.178322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eaf82-cc98-4f7b-991b-79773bd7fc5d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:22.179839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-668eaf82-cc98-4f7b-991b-79773bd7fc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.197374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eaf82-cc98-4f7b-991b-79773bd7fc5d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:22.198565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 987/1965] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:22 2022] PUT /identity/v3/projects/ee47f7edd88042a3a9508e3e24343c86/users/3ade449bab27445693224e6bf1e881ac/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:22.199494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3474c3-b6da-473f-94ea-d0c3874ad521 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:22.201155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7a3474c3-b6da-473f-94ea-d0c3874ad521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.288205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3474c3-b6da-473f-94ea-d0c3874ad521 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:22.292505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87dbaf-c102-4f25-8bf1-494a68f918fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:22.293491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87dbaf-c102-4f25-8bf1-494a68f918fe service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.293775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87dbaf-c102-4f25-8bf1-494a68f918fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.294049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87dbaf-c102-4f25-8bf1-494a68f918fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.296110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99923152-a535-4836-a64a-a29b0926cfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:22.296797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99923152-a535-4836-a64a-a29b0926cfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.297803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99923152-a535-4836-a64a-a29b0926cfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.298155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99923152-a535-4836-a64a-a29b0926cfe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.298870 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99923152-a535-4836-a64a-a29b0926cfe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:22.300824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99923152-a535-4836-a64a-a29b0926cfe3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:22.325538 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:22.334348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87dbaf-c102-4f25-8bf1-494a68f918fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:22.338273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 979/1966] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:22 2022] POST /identity/v3/projects => generated 409 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:22.338273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e4959-7643-4d98-9f57-111056f80144 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.338508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e4959-7643-4d98-9f57-111056f80144 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.339021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e4959-7643-4d98-9f57-111056f80144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.341801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87dbaf-c102-4f25-8bf1-494a68f918fe service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:22.365205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 988/1967] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:22.366359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87dbaf-c102-4f25-8bf1-494a68f918fe service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:22.399686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87dbaf-c102-4f25-8bf1-494a68f918fe service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:22.401663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3f2e4959-7643-4d98-9f57-111056f80144 None None] MFA Rules not processed for user `3ade449bab27445693224e6bf1e881ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:22.406923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3f2e4959-7643-4d98-9f57-111056f80144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.501552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30d9db-dff5-4d76-9bb1-1058bfc1b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:22.502691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30d9db-dff5-4d76-9bb1-1058bfc1b05a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.502976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30d9db-dff5-4d76-9bb1-1058bfc1b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.503537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30d9db-dff5-4d76-9bb1-1058bfc1b05a admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.504455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30d9db-dff5-4d76-9bb1-1058bfc1b05a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:22.507022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30d9db-dff5-4d76-9bb1-1058bfc1b05a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:22.513694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 980/1968] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:22.514610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e4959-7643-4d98-9f57-111056f80144 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:22.520680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3f2e4959-7643-4d98-9f57-111056f80144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.525142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:22.549916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e4959-7643-4d98-9f57-111056f80144 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:22.553928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 989/1969] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:22 2022] POST /identity/v3/users => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:22.555023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30d9db-dff5-4d76-9bb1-1058bfc1b05a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:22.557229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1e30d9db-dff5-4d76-9bb1-1058bfc1b05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.562286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:22.563601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.565750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.566709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] PATH_INFO: `/v3/users/23aea071939d4e80bfc9680293dc518c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.569922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] RBAC: Authorizing `identity:delete_user(user_id=23aea071939d4e80bfc9680293dc518c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:22.572117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:22.666330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30d9db-dff5-4d76-9bb1-1058bfc1b05a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:22.675753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4287b2e-961e-433b-b921-cf9e613a3d80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:22.676444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4287b2e-961e-433b-b921-cf9e613a3d80 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.676747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4287b2e-961e-433b-b921-cf9e613a3d80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.677039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4287b2e-961e-433b-b921-cf9e613a3d80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.683084 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23aea071939d4e80bfc9680293dc518c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:22.696207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4287b2e-961e-433b-b921-cf9e613a3d80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:22.697242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23aea071939d4e80bfc9680293dc518c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:22.704383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4287b2e-961e-433b-b921-cf9e613a3d80 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:22.714665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23aea071939d4e80bfc9680293dc518c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:22.727472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23aea071939d4e80bfc9680293dc518c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:22.729259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6f237418-5a72-4d36-a6a4-1e4cff52d67f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23aea071939d4e80bfc9680293dc518c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:22.729765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 990/1970] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:22.730779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4287b2e-961e-433b-b921-cf9e613a3d80 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:22.742053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 981/1971] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:22 2022] DELETE /identity/v3/users/23aea071939d4e80bfc9680293dc518c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:22.744768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614009b2-56a7-4446-864e-f10c8080c2f6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.745124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614009b2-56a7-4446-864e-f10c8080c2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.745400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614009b2-56a7-4446-864e-f10c8080c2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.757682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4287b2e-961e-433b-b921-cf9e613a3d80 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:22.769553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-614009b2-56a7-4446-864e-f10c8080c2f6 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:22.770843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-614009b2-56a7-4446-864e-f10c8080c2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.843825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 982/1972] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:22.871571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614009b2-56a7-4446-864e-f10c8080c2f6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:22.873113 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-614009b2-56a7-4446-864e-f10c8080c2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:22.878925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:22.915322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7543cb-fc15-4765-b912-9cecc2410f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:22.920515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7543cb-fc15-4765-b912-9cecc2410f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:22.920724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7543cb-fc15-4765-b912-9cecc2410f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:22.924776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7543cb-fc15-4765-b912-9cecc2410f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:22.925693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7543cb-fc15-4765-b912-9cecc2410f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:22.929840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7543cb-fc15-4765-b912-9cecc2410f65 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:22.967510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 991/1973] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:22.971644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7543cb-fc15-4765-b912-9cecc2410f65 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:22.975878 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ad7543cb-fc15-4765-b912-9cecc2410f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:23.009154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7543cb-fc15-4765-b912-9cecc2410f65 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:23.020764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef885dc6-13f2-4c8e-838f-5489dd8d03b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:23.021721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef885dc6-13f2-4c8e-838f-5489dd8d03b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:23.022004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef885dc6-13f2-4c8e-838f-5489dd8d03b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:23.022289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef885dc6-13f2-4c8e-838f-5489dd8d03b5 admin admin] PATH_INFO: `/v3/projects/8aaa0b2e79be4e4893c81bf306fa0b10/users/830edfaff68c49f288320996a41b635c/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:23.028498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614009b2-56a7-4446-864e-f10c8080c2f6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:23.037856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18910415-a669-49a0-adc3-d4cde09b22c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:23.038540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18910415-a669-49a0-adc3-d4cde09b22c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:23.038821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18910415-a669-49a0-adc3-d4cde09b22c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:23.039093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18910415-a669-49a0-adc3-d4cde09b22c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:23.057571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef885dc6-13f2-4c8e-838f-5489dd8d03b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aaa0b2e79be4e4893c81bf306fa0b10, user_id=830edfaff68c49f288320996a41b635c, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:23.060330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef885dc6-13f2-4c8e-838f-5489dd8d03b5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:23.062483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18910415-a669-49a0-adc3-d4cde09b22c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:23.067091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18910415-a669-49a0-adc3-d4cde09b22c4 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:23.097831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 992/1974] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:22 2022] PUT /identity/v3/projects/8aaa0b2e79be4e4893c81bf306fa0b10/users/830edfaff68c49f288320996a41b635c/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:23.102820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef885dc6-13f2-4c8e-838f-5489dd8d03b5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:23.104885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 983/1975] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:23.105619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ef885dc6-13f2-4c8e-838f-5489dd8d03b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:23.106803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18910415-a669-49a0-adc3-d4cde09b22c4 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:23.120445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18910415-a669-49a0-adc3-d4cde09b22c4 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:23.207749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9419d-a925-4e97-bfad-3b837dbea7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:23.208605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9419d-a925-4e97-bfad-3b837dbea7df admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:23.209007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9419d-a925-4e97-bfad-3b837dbea7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:23.209322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef885dc6-13f2-4c8e-838f-5489dd8d03b5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:23.209453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9419d-a925-4e97-bfad-3b837dbea7df admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:23.210418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9419d-a925-4e97-bfad-3b837dbea7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:23.213005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9419d-a925-4e97-bfad-3b837dbea7df admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:23.214546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:23.215762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:23.216050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:23.216334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] PATH_INFO: `/v3/projects/828e02fe01fc4c43b1fd33fd076157d5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:23.217705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828e02fe01fc4c43b1fd33fd076157d5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:23.219848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:23.228287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 984/1976] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:23.229378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:23.232984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9419d-a925-4e97-bfad-3b837dbea7df admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:23.250988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9419d-a925-4e97-bfad-3b837dbea7df admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:23.267030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffca8a7-2d12-4b63-91e5-42724bc1dfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:23.268508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffca8a7-2d12-4b63-91e5-42724bc1dfba admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:23.269014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffca8a7-2d12-4b63-91e5-42724bc1dfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:23.269305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffca8a7-2d12-4b63-91e5-42724bc1dfba admin admin] PATH_INFO: `/v3/projects/8aaa0b2e79be4e4893c81bf306fa0b10/users/830edfaff68c49f288320996a41b635c/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:23.282263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffca8a7-2d12-4b63-91e5-42724bc1dfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aaa0b2e79be4e4893c81bf306fa0b10, user_id=830edfaff68c49f288320996a41b635c, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:23.282263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffca8a7-2d12-4b63-91e5-42724bc1dfba admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:23.307484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828e02fe01fc4c43b1fd33fd076157d5', 'request_id': 'req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.309040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 985/1977] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:23 2022] PUT /identity/v3/projects/8aaa0b2e79be4e4893c81bf306fa0b10/users/830edfaff68c49f288320996a41b635c/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:23.321038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828e02fe01fc4c43b1fd33fd076157d5', 'request_id': 'req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.323560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da55612-4663-4d57-9ab0-9578f3c6817d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:23.325362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da55612-4663-4d57-9ab0-9578f3c6817d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:23.325811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da55612-4663-4d57-9ab0-9578f3c6817d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:23.332857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828e02fe01fc4c43b1fd33fd076157d5', 'request_id': 'req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.357552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] The token cache is being invalidate because project 828e02fe01fc4c43b1fd33fd076157d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:23.357936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.366822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.368581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 993/1978] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:23 2022] DELETE /identity/v3/projects/828e02fe01fc4c43b1fd33fd076157d5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:23.370136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3da55612-4663-4d57-9ab0-9578f3c6817d None None] MFA Rules not processed for user `830edfaff68c49f288320996a41b635c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:23.372397 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3da55612-4663-4d57-9ab0-9578f3c6817d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:23.372826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:23.374596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:23.484812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 986/1979] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:23.495427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cafc13-b0b0-4235-a0d1-c2c1adb61843 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:23.497501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da55612-4663-4d57-9ab0-9578f3c6817d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:23.498915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3da55612-4663-4d57-9ab0-9578f3c6817d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:23.516452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:23.567618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da55612-4663-4d57-9ab0-9578f3c6817d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:23.567932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:23.568849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:23.569185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:23.569403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] PATH_INFO: `/v3/users/cf00bfc5ee3b4f6db45058253890ffa4` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:23.570592 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf00bfc5ee3b4f6db45058253890ffa4)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:23.572566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:23.572840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a77f58-ac8f-4536-bf25-8ae6ce83dabe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:23.573570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a77f58-ac8f-4536-bf25-8ae6ce83dabe service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:23.573965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a77f58-ac8f-4536-bf25-8ae6ce83dabe service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:23.574373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a77f58-ac8f-4536-bf25-8ae6ce83dabe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:23.588512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a77f58-ac8f-4536-bf25-8ae6ce83dabe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:23.590915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a77f58-ac8f-4536-bf25-8ae6ce83dabe service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:23.605119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 987/1980] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:23 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:36:23.626197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:23.628393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf00bfc5ee3b4f6db45058253890ffa4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.643922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf00bfc5ee3b4f6db45058253890ffa4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.655293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf00bfc5ee3b4f6db45058253890ffa4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.655918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf00bfc5ee3b4f6db45058253890ffa4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.666478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf00bfc5ee3b4f6db45058253890ffa4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:23.673831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 994/1981] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:23 2022] DELETE /identity/v3/users/cf00bfc5ee3b4f6db45058253890ffa4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:23.799781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a77f58-ac8f-4536-bf25-8ae6ce83dabe service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:23.809914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a77f58-ac8f-4536-bf25-8ae6ce83dabe service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:23.872623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:23.873449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:23.873824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:23.874188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:23.876041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:23.951073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:23.952056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:23.965116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 988/1982] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:24.165216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:24.166935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:24.254177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:24.255725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:24.293783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725b148-174b-4be1-92d7-311cbab53848 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:24.301274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:24.303496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:24.303864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:24.304150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] PATH_INFO: `/v3/projects/f219fe4790504a588953fa0756c6edd0` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:24.305978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f219fe4790504a588953fa0756c6edd0)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:24.310064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:24.426790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da546ea0-f7bb-4956-af1e-057879d02cc9 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:24.466860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:24.477765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f219fe4790504a588953fa0756c6edd0', 'request_id': 'req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.521024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f219fe4790504a588953fa0756c6edd0', 'request_id': 'req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.533243 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f219fe4790504a588953fa0756c6edd0', 'request_id': 'req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.565633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] The token cache is being invalidate because project f219fe4790504a588953fa0756c6edd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:24.566844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.568254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.571806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 995/1983] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:24 2022] DELETE /identity/v3/projects/f219fe4790504a588953fa0756c6edd0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:24.574484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:24.575706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:24.575828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:24.576219 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:24.577688 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] PATH_INFO: `/v3/projects/5453a671afab49988f9fc38fb7a77a46` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:24.581484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:24.583375 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5453a671afab49988f9fc38fb7a77a46)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:24.585520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:24.613117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0b32c-8fef-44f3-ba41-c77e0578d4fc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:24.625486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:24.629913 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:24.630450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:24.631659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] PATH_INFO: `/v3/users/da1b665ca63d428f93fb2a1dbb5f02db` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:24.632375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1b665ca63d428f93fb2a1dbb5f02db)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:24.637918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:24.670591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5453a671afab49988f9fc38fb7a77a46', 'request_id': 'req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.681479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5453a671afab49988f9fc38fb7a77a46', 'request_id': 'req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.696134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5453a671afab49988f9fc38fb7a77a46', 'request_id': 'req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.729439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] The token cache is being invalidate because project 5453a671afab49988f9fc38fb7a77a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:24.731584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.733674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.735621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 989/1984] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:24 2022] DELETE /identity/v3/projects/5453a671afab49988f9fc38fb7a77a46 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:24.740474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:24.741164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1b665ca63d428f93fb2a1dbb5f02db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.743695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:24.746617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:24.771563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1b665ca63d428f93fb2a1dbb5f02db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.784646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1b665ca63d428f93fb2a1dbb5f02db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.785430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1b665ca63d428f93fb2a1dbb5f02db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.803934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1b665ca63d428f93fb2a1dbb5f02db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:24.813121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 996/1985] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:24 2022] DELETE /identity/v3/users/da1b665ca63d428f93fb2a1dbb5f02db => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:36:24.970389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb496ebb-1f8d-4d04-8144-2b7ec94410d9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:25.046313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:25.059917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:25.060927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:25.061377 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:25.061871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] PATH_INFO: `/v3/users/6ec63a56873d4abe8cf9e4f5b4717b49` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:25.063811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec63a56873d4abe8cf9e4f5b4717b49)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:25.067763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:25.142875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec63a56873d4abe8cf9e4f5b4717b49'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.157763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec63a56873d4abe8cf9e4f5b4717b49'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.173371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec63a56873d4abe8cf9e4f5b4717b49'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.187315 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec63a56873d4abe8cf9e4f5b4717b49'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.189275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0af54d4d-d7ae-44df-9549-a985d6acfce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec63a56873d4abe8cf9e4f5b4717b49'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.200578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 990/1986] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:24 2022] DELETE /identity/v3/users/6ec63a56873d4abe8cf9e4f5b4717b49 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:25.299685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:25.301280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:25.320175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bcbf6-242d-4d1c-a77a-fc4159687e46 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:25.403183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:25.404807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:25.404807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:25.406177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] PATH_INFO: `/v3/projects/8f26550df6774d4c819f5e29d4a761ea` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:25.408643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f26550df6774d4c819f5e29d4a761ea)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:25.411226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:25.489144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f26550df6774d4c819f5e29d4a761ea', 'request_id': 'req-77ec74d2-427f-448e-a411-504cbd323488'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.505757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f26550df6774d4c819f5e29d4a761ea', 'request_id': 'req-77ec74d2-427f-448e-a411-504cbd323488'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.514791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f26550df6774d4c819f5e29d4a761ea', 'request_id': 'req-77ec74d2-427f-448e-a411-504cbd323488'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.537040 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] The token cache is being invalidate because project 8f26550df6774d4c819f5e29d4a761ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:25.537434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.538182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77ec74d2-427f-448e-a411-504cbd323488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.543160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 997/1987] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:25 2022] DELETE /identity/v3/projects/8f26550df6774d4c819f5e29d4a761ea => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:25.545545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa5490-60bc-436b-8c59-52030fb66e8f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:25.546003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa5490-60bc-436b-8c59-52030fb66e8f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:25.546003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa5490-60bc-436b-8c59-52030fb66e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:25.576240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c5fa5490-60bc-436b-8c59-52030fb66e8f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:25.577631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c5fa5490-60bc-436b-8c59-52030fb66e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:25.689867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 991/1988] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:25.697708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e78cc0-652a-4d83-b463-3bed69aacb4f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:25.698095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e78cc0-652a-4d83-b463-3bed69aacb4f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:25.698467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e78cc0-652a-4d83-b463-3bed69aacb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:25.733551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-85e78cc0-652a-4d83-b463-3bed69aacb4f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:25.737162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-85e78cc0-652a-4d83-b463-3bed69aacb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:25.740727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa5490-60bc-436b-8c59-52030fb66e8f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:25.743769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c5fa5490-60bc-436b-8c59-52030fb66e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:25.765349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa5490-60bc-436b-8c59-52030fb66e8f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:25.770082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 998/1989] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:25.771246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e78cc0-652a-4d83-b463-3bed69aacb4f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:25.784363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e78cc0-652a-4d83-b463-3bed69aacb4f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:25.788382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:25.789533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:25.789925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:25.791275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] PATH_INFO: `/v3/projects/b484c127324b4b6781258c76f1dc19e6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:25.791642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fa10f-cdf8-46a9-be81-738b451800fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:25.792485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fa10f-cdf8-46a9-be81-738b451800fb admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:25.792960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fa10f-cdf8-46a9-be81-738b451800fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:25.793411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fa10f-cdf8-46a9-be81-738b451800fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:25.794439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fa10f-cdf8-46a9-be81-738b451800fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:25.794920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fa10f-cdf8-46a9-be81-738b451800fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:25.795271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b484c127324b4b6781258c76f1dc19e6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:25.797278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fa10f-cdf8-46a9-be81-738b451800fb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:25.797635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:25.818419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 999/1990] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:25.819867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fa10f-cdf8-46a9-be81-738b451800fb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:25.836925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fa10f-cdf8-46a9-be81-738b451800fb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:25.849358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80098f0c-f8bd-4a41-a6e6-fccc2ec8161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:25.852011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80098f0c-f8bd-4a41-a6e6-fccc2ec8161e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:25.852355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80098f0c-f8bd-4a41-a6e6-fccc2ec8161e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:25.852640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80098f0c-f8bd-4a41-a6e6-fccc2ec8161e admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:25.854187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80098f0c-f8bd-4a41-a6e6-fccc2ec8161e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:25.857060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80098f0c-f8bd-4a41-a6e6-fccc2ec8161e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:25.884156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b484c127324b4b6781258c76f1dc19e6', 'request_id': 'req-d2ab03a1-817e-4bca-8541-66b3918a6e0e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.893013 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b484c127324b4b6781258c76f1dc19e6', 'request_id': 'req-d2ab03a1-817e-4bca-8541-66b3918a6e0e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.902320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1000/1991] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:25 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:25.906919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80098f0c-f8bd-4a41-a6e6-fccc2ec8161e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:25.909645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b484c127324b4b6781258c76f1dc19e6', 'request_id': 'req-d2ab03a1-817e-4bca-8541-66b3918a6e0e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.922115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:25.924174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:25.927184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] The token cache is being invalidate because project b484c127324b4b6781258c76f1dc19e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:25.927641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.928522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2ab03a1-817e-4bca-8541-66b3918a6e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:25.931031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 992/1992] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:25 2022] DELETE /identity/v3/projects/b484c127324b4b6781258c76f1dc19e6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:25.939508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f5520-351b-443a-ad54-a80ad72163d0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:25.939508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f5520-351b-443a-ad54-a80ad72163d0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:25.939508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f5520-351b-443a-ad54-a80ad72163d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:25.966163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fc1f5520-351b-443a-ad54-a80ad72163d0 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:25.968215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fc1f5520-351b-443a-ad54-a80ad72163d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:26.014746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80098f0c-f8bd-4a41-a6e6-fccc2ec8161e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.025153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa154b4-edab-41a8-a408-ac267f0392c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.025945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa154b4-edab-41a8-a408-ac267f0392c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.026296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa154b4-edab-41a8-a408-ac267f0392c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.026950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa154b4-edab-41a8-a408-ac267f0392c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.027814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa154b4-edab-41a8-a408-ac267f0392c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.030773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa154b4-edab-41a8-a408-ac267f0392c1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.051588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 993/1993] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:26.061166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc6084-6267-4756-91f1-7773b7620811 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.062380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc6084-6267-4756-91f1-7773b7620811 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.062799 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc6084-6267-4756-91f1-7773b7620811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.063147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1001/1994] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:25 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:26.067171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa154b4-edab-41a8-a408-ac267f0392c1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.068894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7aa154b4-edab-41a8-a408-ac267f0392c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:26.081368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa154b4-edab-41a8-a408-ac267f0392c1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.085985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c047c4-ce89-44e2-a3ee-1d5b44ed83d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.086118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-75dc6084-6267-4756-91f1-7773b7620811 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:26.086657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c047c4-ce89-44e2-a3ee-1d5b44ed83d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.086941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c047c4-ce89-44e2-a3ee-1d5b44ed83d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.087215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c047c4-ce89-44e2-a3ee-1d5b44ed83d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.087599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-75dc6084-6267-4756-91f1-7773b7620811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:26.087944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c047c4-ce89-44e2-a3ee-1d5b44ed83d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.089606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c047c4-ce89-44e2-a3ee-1d5b44ed83d9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.100673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 994/1995] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:26.104554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1002/1996] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:26.105265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc6084-6267-4756-91f1-7773b7620811 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.105492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c047c4-ce89-44e2-a3ee-1d5b44ed83d9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.116242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c047c4-ce89-44e2-a3ee-1d5b44ed83d9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.117487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc6084-6267-4756-91f1-7773b7620811 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.122196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef39ee-003d-424b-aad4-20ee3efd1864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.122329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c25d-fd47-4737-ab18-3fd2f29bdb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.122870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef39ee-003d-424b-aad4-20ee3efd1864 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.122990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c25d-fd47-4737-ab18-3fd2f29bdb2a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.123151 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef39ee-003d-424b-aad4-20ee3efd1864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.123286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c25d-fd47-4737-ab18-3fd2f29bdb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.123415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef39ee-003d-424b-aad4-20ee3efd1864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.123701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c25d-fd47-4737-ab18-3fd2f29bdb2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.124149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef39ee-003d-424b-aad4-20ee3efd1864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.124548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c25d-fd47-4737-ab18-3fd2f29bdb2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:26.124944 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c25d-fd47-4737-ab18-3fd2f29bdb2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.125797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef39ee-003d-424b-aad4-20ee3efd1864 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.126721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c25d-fd47-4737-ab18-3fd2f29bdb2a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.140089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1003/1997] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:26.143304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef39ee-003d-424b-aad4-20ee3efd1864 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.144635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 995/1998] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:26.150294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c25d-fd47-4737-ab18-3fd2f29bdb2a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.161513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef39ee-003d-424b-aad4-20ee3efd1864 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.161854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c25d-fd47-4737-ab18-3fd2f29bdb2a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.166982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f95e5a-adad-476b-9c41-e3de9637ffed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.167188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471d880-4d2b-4577-b436-064e4eaea135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.167843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471d880-4d2b-4577-b436-064e4eaea135 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.168014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f95e5a-adad-476b-9c41-e3de9637ffed admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.168111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471d880-4d2b-4577-b436-064e4eaea135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.168375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f95e5a-adad-476b-9c41-e3de9637ffed admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.168497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471d880-4d2b-4577-b436-064e4eaea135 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.168650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f95e5a-adad-476b-9c41-e3de9637ffed admin admin] PATH_INFO: `/v3/projects/a9c23abad4994701a4153b3d7ba12645/users/b43b338b0bca4764a0bf0de2b2e882f1/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.169097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471d880-4d2b-4577-b436-064e4eaea135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.171085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471d880-4d2b-4577-b436-064e4eaea135 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.186611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f95e5a-adad-476b-9c41-e3de9637ffed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c23abad4994701a4153b3d7ba12645, user_id=b43b338b0bca4764a0bf0de2b2e882f1, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.189911 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f95e5a-adad-476b-9c41-e3de9637ffed admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.200825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 996/1999] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:26 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:26.205945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471d880-4d2b-4577-b436-064e4eaea135 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.206831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1004/2000] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:26 2022] PUT /identity/v3/projects/a9c23abad4994701a4153b3d7ba12645/users/b43b338b0bca4764a0bf0de2b2e882f1/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:26.218107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb6af0-72d9-419c-adf0-d8e6e5ca4be3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.218394 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb6af0-72d9-419c-adf0-d8e6e5ca4be3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.218679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb6af0-72d9-419c-adf0-d8e6e5ca4be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.247760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f7fb6af0-72d9-419c-adf0-d8e6e5ca4be3 None None] MFA Rules not processed for user `b43b338b0bca4764a0bf0de2b2e882f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:26.249047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f7fb6af0-72d9-419c-adf0-d8e6e5ca4be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:26.283834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471d880-4d2b-4577-b436-064e4eaea135 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.325268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1005/2001] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:26.330388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb6af0-72d9-419c-adf0-d8e6e5ca4be3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.347468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99410f1-1752-46ab-b87b-020f28564629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.349075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99410f1-1752-46ab-b87b-020f28564629 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.349075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99410f1-1752-46ab-b87b-020f28564629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.349075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99410f1-1752-46ab-b87b-020f28564629 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.349657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99410f1-1752-46ab-b87b-020f28564629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.351694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99410f1-1752-46ab-b87b-020f28564629 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.362575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:26.386611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:26.388817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 997/2002] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:26 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:36:26.393174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99410f1-1752-46ab-b87b-020f28564629 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.403502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99410f1-1752-46ab-b87b-020f28564629 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.409397 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af879cc-6303-4bb9-8c61-afb3321c917a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.409946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb6af0-72d9-419c-adf0-d8e6e5ca4be3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.410095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af879cc-6303-4bb9-8c61-afb3321c917a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.410344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af879cc-6303-4bb9-8c61-afb3321c917a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.410620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af879cc-6303-4bb9-8c61-afb3321c917a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.411356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af879cc-6303-4bb9-8c61-afb3321c917a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.412997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af879cc-6303-4bb9-8c61-afb3321c917a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.414503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb6b6a-2ec7-47fd-b580-c9fcb5cf6607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.415189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb6b6a-2ec7-47fd-b580-c9fcb5cf6607 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.415471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb6b6a-2ec7-47fd-b580-c9fcb5cf6607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.415775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb6b6a-2ec7-47fd-b580-c9fcb5cf6607 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.416478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb6b6a-2ec7-47fd-b580-c9fcb5cf6607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.418419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb6b6a-2ec7-47fd-b580-c9fcb5cf6607 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.424783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 998/2003] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:26.428158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af879cc-6303-4bb9-8c61-afb3321c917a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.438360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af879cc-6303-4bb9-8c61-afb3321c917a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.448764 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19d22-968e-4c54-81e3-8f150ce40d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.449940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19d22-968e-4c54-81e3-8f150ce40d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.450083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1006/2004] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:26 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:26.453126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19d22-968e-4c54-81e3-8f150ce40d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.454754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19d22-968e-4c54-81e3-8f150ce40d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.455554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea19d22-968e-4c54-81e3-8f150ce40d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.457341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea19d22-968e-4c54-81e3-8f150ce40d50 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.458058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb6b6a-2ec7-47fd-b580-c9fcb5cf6607 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.470854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 999/2005] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:26.476364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19d22-968e-4c54-81e3-8f150ce40d50 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.489074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19d22-968e-4c54-81e3-8f150ce40d50 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.529805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb6b6a-2ec7-47fd-b580-c9fcb5cf6607 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.537265 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a86d9-749a-41d8-b903-384452569e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.537880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a86d9-749a-41d8-b903-384452569e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.538165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a86d9-749a-41d8-b903-384452569e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.538443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a86d9-749a-41d8-b903-384452569e9a admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.539152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a86d9-749a-41d8-b903-384452569e9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.541544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a86d9-749a-41d8-b903-384452569e9a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.558464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290730b-b136-42d2-98b4-58e8af13e8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.559625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290730b-b136-42d2-98b4-58e8af13e8df admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.560032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290730b-b136-42d2-98b4-58e8af13e8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.560585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290730b-b136-42d2-98b4-58e8af13e8df admin admin] PATH_INFO: `/v3/projects/9513ee5af8e84d48a7f28b4c77db2dd0/users/57e63c7198d2450698858f9b84393c43/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.565645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1007/2006] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:26 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:26.569568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a86d9-749a-41d8-b903-384452569e9a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.580109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290730b-b136-42d2-98b4-58e8af13e8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=9513ee5af8e84d48a7f28b4c77db2dd0, user_id=57e63c7198d2450698858f9b84393c43, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.582969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290730b-b136-42d2-98b4-58e8af13e8df admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.593363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a86d9-749a-41d8-b903-384452569e9a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.599421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80554303-16c7-45dd-857c-280f828a4494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.600106 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80554303-16c7-45dd-857c-280f828a4494 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.600433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80554303-16c7-45dd-857c-280f828a4494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.600714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80554303-16c7-45dd-857c-280f828a4494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.601644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80554303-16c7-45dd-857c-280f828a4494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.603311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80554303-16c7-45dd-857c-280f828a4494 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.609244 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1000/2007] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:26 2022] PUT /identity/v3/projects/9513ee5af8e84d48a7f28b4c77db2dd0/users/57e63c7198d2450698858f9b84393c43/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:26.620777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1008/2008] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:26.623068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069d8dd-2a69-4352-b846-0d038dc3396c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.623572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069d8dd-2a69-4352-b846-0d038dc3396c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.623694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069d8dd-2a69-4352-b846-0d038dc3396c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.624844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80554303-16c7-45dd-857c-280f828a4494 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.639926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80554303-16c7-45dd-857c-280f828a4494 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.644616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c069d8dd-2a69-4352-b846-0d038dc3396c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:26.646095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c069d8dd-2a69-4352-b846-0d038dc3396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:26.705686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb45ce6-cb29-4fcd-a4b0-a198f430cacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.706738 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb45ce6-cb29-4fcd-a4b0-a198f430cacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.707087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb45ce6-cb29-4fcd-a4b0-a198f430cacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.707451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb45ce6-cb29-4fcd-a4b0-a198f430cacf admin admin] PATH_INFO: `/v3/projects/1f14e9fa69524675aad110fe745ff08d/users/417e33e21a264c5fa9fc78e05d156297/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.720491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1001/2009] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:26.724144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb45ce6-cb29-4fcd-a4b0-a198f430cacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f14e9fa69524675aad110fe745ff08d, user_id=417e33e21a264c5fa9fc78e05d156297, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.726694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb45ce6-cb29-4fcd-a4b0-a198f430cacf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.731282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069d8dd-2a69-4352-b846-0d038dc3396c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.732810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c069d8dd-2a69-4352-b846-0d038dc3396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:26.744089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1009/2010] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:26 2022] PUT /identity/v3/projects/1f14e9fa69524675aad110fe745ff08d/users/417e33e21a264c5fa9fc78e05d156297/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:26.748028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb45ce6-cb29-4fcd-a4b0-a198f430cacf admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.803608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:26.806125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069d8dd-2a69-4352-b846-0d038dc3396c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.817842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:26.848593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb45ce6-cb29-4fcd-a4b0-a198f430cacf admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.854192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3218c8-c02c-47e2-8d58-ad2f6e8ad5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.854889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3218c8-c02c-47e2-8d58-ad2f6e8ad5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.855413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3218c8-c02c-47e2-8d58-ad2f6e8ad5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.856090 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3218c8-c02c-47e2-8d58-ad2f6e8ad5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.857364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3218c8-c02c-47e2-8d58-ad2f6e8ad5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.858799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3218c8-c02c-47e2-8d58-ad2f6e8ad5c3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.874717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89127fab-771c-4303-9189-714fe81d3133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.875384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127fab-771c-4303-9189-714fe81d3133 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.875694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127fab-771c-4303-9189-714fe81d3133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.875965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127fab-771c-4303-9189-714fe81d3133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.883153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1010/2011] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:26.884416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3218c8-c02c-47e2-8d58-ad2f6e8ad5c3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:26.891589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89127fab-771c-4303-9189-714fe81d3133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.893835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89127fab-771c-4303-9189-714fe81d3133 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.897850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3218c8-c02c-47e2-8d58-ad2f6e8ad5c3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:26.902862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fe1ec-afdc-4a9d-a899-c71b39bd5718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:26.903765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fe1ec-afdc-4a9d-a899-c71b39bd5718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.904049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fe1ec-afdc-4a9d-a899-c71b39bd5718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.904324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fe1ec-afdc-4a9d-a899-c71b39bd5718 admin admin] PATH_INFO: `/v3/projects/1f14e9fa69524675aad110fe745ff08d/users/417e33e21a264c5fa9fc78e05d156297/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.905466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1002/2012] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:36:26.906435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fe1ec-afdc-4a9d-a899-c71b39bd5718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f14e9fa69524675aad110fe745ff08d, user_id=417e33e21a264c5fa9fc78e05d156297, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:26.909416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fe1ec-afdc-4a9d-a899-c71b39bd5718 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:26.924144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1011/2013] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:26 2022] PUT /identity/v3/projects/1f14e9fa69524675aad110fe745ff08d/users/417e33e21a264c5fa9fc78e05d156297/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:26.934426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed354-c1d6-45d2-bb9f-0725621646ed None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:26.934725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed354-c1d6-45d2-bb9f-0725621646ed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:26.934995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed354-c1d6-45d2-bb9f-0725621646ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:26.964198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fc8ed354-c1d6-45d2-bb9f-0725621646ed None None] MFA Rules not processed for user `417e33e21a264c5fa9fc78e05d156297`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:26.965814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fc8ed354-c1d6-45d2-bb9f-0725621646ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:27.085146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1003/2014] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:27.096914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fe1ec-afdc-4a9d-a899-c71b39bd5718 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:27.098543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0b1fe1ec-afdc-4a9d-a899-c71b39bd5718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:27.189321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fe1ec-afdc-4a9d-a899-c71b39bd5718 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:27.193339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90195970-22cc-4bbd-9088-fec4cd2cb79d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:27.194008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90195970-22cc-4bbd-9088-fec4cd2cb79d service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:27.194289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90195970-22cc-4bbd-9088-fec4cd2cb79d service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:27.194564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90195970-22cc-4bbd-9088-fec4cd2cb79d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:27.206062 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90195970-22cc-4bbd-9088-fec4cd2cb79d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:27.207998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90195970-22cc-4bbd-9088-fec4cd2cb79d service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:27.221554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1012/2015] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:27.380627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed354-c1d6-45d2-bb9f-0725621646ed None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:27.392521 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed354-c1d6-45d2-bb9f-0725621646ed None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:27.399717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd9c2f-7f3b-4a9b-82a0-666d7b2d0d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:27.400779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd9c2f-7f3b-4a9b-82a0-666d7b2d0d83 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:27.401253 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd9c2f-7f3b-4a9b-82a0-666d7b2d0d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:27.401591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd9c2f-7f3b-4a9b-82a0-666d7b2d0d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:27.476059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd9c2f-7f3b-4a9b-82a0-666d7b2d0d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:27.478537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd9c2f-7f3b-4a9b-82a0-666d7b2d0d83 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:27.497089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1004/2016] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:27.637629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1caf56-a4f7-4f9c-9655-cb1d27ef69c6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:27.638230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1caf56-a4f7-4f9c-9655-cb1d27ef69c6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:27.638405 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1caf56-a4f7-4f9c-9655-cb1d27ef69c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:27.678788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ad1caf56-a4f7-4f9c-9655-cb1d27ef69c6 None None] MFA Rules not processed for user `57e63c7198d2450698858f9b84393c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:27.680137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ad1caf56-a4f7-4f9c-9655-cb1d27ef69c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:27.769563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1013/2017] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:27.776863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd9c2f-7f3b-4a9b-82a0-666d7b2d0d83 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:27.778666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f2cd9c2f-7f3b-4a9b-82a0-666d7b2d0d83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:27.799655 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd9c2f-7f3b-4a9b-82a0-666d7b2d0d83 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:27.809461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fe495-c9ba-4bbb-95f4-bd611a381253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:27.810145 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fe495-c9ba-4bbb-95f4-bd611a381253 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:27.810432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fe495-c9ba-4bbb-95f4-bd611a381253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:27.810712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fe495-c9ba-4bbb-95f4-bd611a381253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:27.829122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fe495-c9ba-4bbb-95f4-bd611a381253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:27.830620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fe495-c9ba-4bbb-95f4-bd611a381253 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:27.841540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1005/2018] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:35.593353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5639b6-14ea-43a5-b237-c9d80aaa5419 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:35.594609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5639b6-14ea-43a5-b237-c9d80aaa5419 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:35.594904 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5639b6-14ea-43a5-b237-c9d80aaa5419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:35.629859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8f5639b6-14ea-43a5-b237-c9d80aaa5419 None None] MFA Rules not processed for user `57e63c7198d2450698858f9b84393c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:35.632780 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8f5639b6-14ea-43a5-b237-c9d80aaa5419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:35.656807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1014/2019] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:36.453918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fe495-c9ba-4bbb-95f4-bd611a381253 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:36.484601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:36.525207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fe495-c9ba-4bbb-95f4-bd611a381253 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:36.529776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:36.530662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:36.531016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:36.531209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] PATH_INFO: `/v3/users/b43b338b0bca4764a0bf0de2b2e882f1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:36.532393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b43b338b0bca4764a0bf0de2b2e882f1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:36.534354 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:36.593419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b43b338b0bca4764a0bf0de2b2e882f1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:36.607855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b43b338b0bca4764a0bf0de2b2e882f1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:36.616746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b43b338b0bca4764a0bf0de2b2e882f1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:36.623757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b43b338b0bca4764a0bf0de2b2e882f1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:36.624398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b43b338b0bca4764a0bf0de2b2e882f1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:36.630392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1006/2020] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:36 2022] DELETE /identity/v3/users/b43b338b0bca4764a0bf0de2b2e882f1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:36.635007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010108ed-ab02-4fa5-bfbd-d03bcce18d76 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:36.635469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010108ed-ab02-4fa5-bfbd-d03bcce18d76 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:36.635576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010108ed-ab02-4fa5-bfbd-d03bcce18d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:36.652943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-010108ed-ab02-4fa5-bfbd-d03bcce18d76 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:36.654141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-010108ed-ab02-4fa5-bfbd-d03bcce18d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:36.693984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:36.718136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1015/2021] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:36.728492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:36.729999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:36.794790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e15d2-8c72-4c5b-a25e-e9892e03f1c4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:36.798776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85eaeab-ce02-41af-aa7a-38833d8c88df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:36.799438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85eaeab-ce02-41af-aa7a-38833d8c88df service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:36.799721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85eaeab-ce02-41af-aa7a-38833d8c88df service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:36.799997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85eaeab-ce02-41af-aa7a-38833d8c88df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:36.810234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85eaeab-ce02-41af-aa7a-38833d8c88df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:36.811657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85eaeab-ce02-41af-aa7a-38833d8c88df service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:36.821093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1007/2022] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:37.096001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010108ed-ab02-4fa5-bfbd-d03bcce18d76 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:37.097681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-010108ed-ab02-4fa5-bfbd-d03bcce18d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:37.111075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010108ed-ab02-4fa5-bfbd-d03bcce18d76 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:37.116909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:37.118899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:37.119198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:37.119495 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] PATH_INFO: `/v3/projects/a9c23abad4994701a4153b3d7ba12645` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:37.120699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9c23abad4994701a4153b3d7ba12645)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:37.122736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:37.182893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9c23abad4994701a4153b3d7ba12645', 'request_id': 'req-9bb21ad3-2903-4291-ab42-53dcb6292abc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.191868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9c23abad4994701a4153b3d7ba12645', 'request_id': 'req-9bb21ad3-2903-4291-ab42-53dcb6292abc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.197931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9c23abad4994701a4153b3d7ba12645', 'request_id': 'req-9bb21ad3-2903-4291-ab42-53dcb6292abc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.209721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] The token cache is being invalidate because project a9c23abad4994701a4153b3d7ba12645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:37.210064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.210683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.212268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1016/2023] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:37 2022] DELETE /identity/v3/projects/a9c23abad4994701a4153b3d7ba12645 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:37.216617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85eaeab-ce02-41af-aa7a-38833d8c88df service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:37.226902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85eaeab-ce02-41af-aa7a-38833d8c88df service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:37.275260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:37.291975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:37.292909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:37.293195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:37.293475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] PATH_INFO: `/v3/users/417e33e21a264c5fa9fc78e05d156297` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:37.294708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=417e33e21a264c5fa9fc78e05d156297)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:37.296622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:37.349021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '417e33e21a264c5fa9fc78e05d156297'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.361968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '417e33e21a264c5fa9fc78e05d156297'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.371761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '417e33e21a264c5fa9fc78e05d156297'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.379632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '417e33e21a264c5fa9fc78e05d156297'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.380320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42a5dd9d-1b42-4436-8e76-90e718d5a2dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '417e33e21a264c5fa9fc78e05d156297'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.386490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1008/2024] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:37 2022] DELETE /identity/v3/users/417e33e21a264c5fa9fc78e05d156297 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:37.667398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:37.668996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:37.683828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb21ad3-2903-4291-ab42-53dcb6292abc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:37.745441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:37.746224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:37.746503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:37.746778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] PATH_INFO: `/v3/projects/1f14e9fa69524675aad110fe745ff08d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:37.748529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f14e9fa69524675aad110fe745ff08d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:37.751602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:37.802857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f14e9fa69524675aad110fe745ff08d', 'request_id': 'req-fe4a732c-87d3-490b-80f6-6f541677c93f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.817000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f14e9fa69524675aad110fe745ff08d', 'request_id': 'req-fe4a732c-87d3-490b-80f6-6f541677c93f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.823962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f14e9fa69524675aad110fe745ff08d', 'request_id': 'req-fe4a732c-87d3-490b-80f6-6f541677c93f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.826944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:37.840662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] The token cache is being invalidate because project 1f14e9fa69524675aad110fe745ff08d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:37.841013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.841822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe4a732c-87d3-490b-80f6-6f541677c93f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:37.843999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1017/2025] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:37 2022] DELETE /identity/v3/projects/1f14e9fa69524675aad110fe745ff08d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:37.849385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73040a6-5096-40b8-9287-6120ee4f971b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:37.849878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73040a6-5096-40b8-9287-6120ee4f971b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:37.849994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73040a6-5096-40b8-9287-6120ee4f971b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:37.868961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c73040a6-5096-40b8-9287-6120ee4f971b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:37.870305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c73040a6-5096-40b8-9287-6120ee4f971b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:37.940570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1009/2026] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:37.949675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11303fd1-3d6d-4590-8fae-12e4779bb35a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:37.949963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11303fd1-3d6d-4590-8fae-12e4779bb35a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:37.950244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11303fd1-3d6d-4590-8fae-12e4779bb35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:37.969530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-11303fd1-3d6d-4590-8fae-12e4779bb35a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:37.971648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-11303fd1-3d6d-4590-8fae-12e4779bb35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:37.979114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1018/2027] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:37.983446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73040a6-5096-40b8-9287-6120ee4f971b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:37.994144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73040a6-5096-40b8-9287-6120ee4f971b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:37.998736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ca3c3-2c5c-45cf-859d-480f3d719016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:37.999345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ca3c3-2c5c-45cf-859d-480f3d719016 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:37.999625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ca3c3-2c5c-45cf-859d-480f3d719016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:37.999905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ca3c3-2c5c-45cf-859d-480f3d719016 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.000644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ca3c3-2c5c-45cf-859d-480f3d719016 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:38.000989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ca3c3-2c5c-45cf-859d-480f3d719016 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.002593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ca3c3-2c5c-45cf-859d-480f3d719016 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.014766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1010/2028] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:38.019096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11303fd1-3d6d-4590-8fae-12e4779bb35a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.029187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11303fd1-3d6d-4590-8fae-12e4779bb35a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.033747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa13116-517b-48a9-b6a7-d60b94a79146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.034393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa13116-517b-48a9-b6a7-d60b94a79146 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.034676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa13116-517b-48a9-b6a7-d60b94a79146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.034948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa13116-517b-48a9-b6a7-d60b94a79146 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.035777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa13116-517b-48a9-b6a7-d60b94a79146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.037741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa13116-517b-48a9-b6a7-d60b94a79146 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.065614 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1019/2029] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:38 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:38.070456 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ca3c3-2c5c-45cf-859d-480f3d719016 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.085246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ca3c3-2c5c-45cf-859d-480f3d719016 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.134565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:38.146049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ed5e1-0330-4f8d-9cfb-8d55f0eb6415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.146741 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ed5e1-0330-4f8d-9cfb-8d55f0eb6415 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.147860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ed5e1-0330-4f8d-9cfb-8d55f0eb6415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.148303 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ed5e1-0330-4f8d-9cfb-8d55f0eb6415 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.149285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ed5e1-0330-4f8d-9cfb-8d55f0eb6415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.151272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ed5e1-0330-4f8d-9cfb-8d55f0eb6415 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.175656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1011/2030] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:38 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:38.179860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa13116-517b-48a9-b6a7-d60b94a79146 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.190152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa13116-517b-48a9-b6a7-d60b94a79146 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.195477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49189eee-ac4e-47cc-9af5-c74f82b29db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.196143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49189eee-ac4e-47cc-9af5-c74f82b29db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.196398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49189eee-ac4e-47cc-9af5-c74f82b29db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.196677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49189eee-ac4e-47cc-9af5-c74f82b29db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.197443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49189eee-ac4e-47cc-9af5-c74f82b29db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.199051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49189eee-ac4e-47cc-9af5-c74f82b29db6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.210643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1020/2031] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:38.214824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ed5e1-0330-4f8d-9cfb-8d55f0eb6415 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.225770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ed5e1-0330-4f8d-9cfb-8d55f0eb6415 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.231700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088c96c-0792-43cb-a44b-a54f767b45d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.232360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088c96c-0792-43cb-a44b-a54f767b45d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.232644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088c96c-0792-43cb-a44b-a54f767b45d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.232957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088c96c-0792-43cb-a44b-a54f767b45d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.233710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088c96c-0792-43cb-a44b-a54f767b45d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.235308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088c96c-0792-43cb-a44b-a54f767b45d0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.246451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1012/2032] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:38.250629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49189eee-ac4e-47cc-9af5-c74f82b29db6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.261195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49189eee-ac4e-47cc-9af5-c74f82b29db6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.266247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82c0d9-fb7c-4017-a5a3-4b4496f011bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.267165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82c0d9-fb7c-4017-a5a3-4b4496f011bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.267480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82c0d9-fb7c-4017-a5a3-4b4496f011bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.267705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82c0d9-fb7c-4017-a5a3-4b4496f011bf admin admin] PATH_INFO: `/v3/projects/5c004f9c67094294be15e2cbc4f6db2e/users/e11357bda6cb4b7a9cd59836e831c353/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.284402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82c0d9-fb7c-4017-a5a3-4b4496f011bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c004f9c67094294be15e2cbc4f6db2e, user_id=e11357bda6cb4b7a9cd59836e831c353, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.287139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82c0d9-fb7c-4017-a5a3-4b4496f011bf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.301016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1021/2033] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:38 2022] PUT /identity/v3/projects/5c004f9c67094294be15e2cbc4f6db2e/users/e11357bda6cb4b7a9cd59836e831c353/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:38.315701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db8126-fb31-42eb-838d-ce09e00f197b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.315994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db8126-fb31-42eb-838d-ce09e00f197b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.316273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db8126-fb31-42eb-838d-ce09e00f197b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.344433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c9db8126-fb31-42eb-838d-ce09e00f197b None None] MFA Rules not processed for user `e11357bda6cb4b7a9cd59836e831c353`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:38.345670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c9db8126-fb31-42eb-838d-ce09e00f197b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:38.416934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1013/2034] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:38.421586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82c0d9-fb7c-4017-a5a3-4b4496f011bf admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.494081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82c0d9-fb7c-4017-a5a3-4b4496f011bf admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.499816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2500bc-c8d2-4240-9382-0772182c72ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.500490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2500bc-c8d2-4240-9382-0772182c72ec admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.500844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2500bc-c8d2-4240-9382-0772182c72ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.501066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2500bc-c8d2-4240-9382-0772182c72ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.501761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2500bc-c8d2-4240-9382-0772182c72ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.503774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2500bc-c8d2-4240-9382-0772182c72ec admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.533485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1022/2035] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:38 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:38.537914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db8126-fb31-42eb-838d-ce09e00f197b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.548721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db8126-fb31-42eb-838d-ce09e00f197b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.618608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cc1e1-c5a6-4245-a2db-6725f42aa6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.619299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cc1e1-c5a6-4245-a2db-6725f42aa6ba admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.619610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cc1e1-c5a6-4245-a2db-6725f42aa6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.619958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cc1e1-c5a6-4245-a2db-6725f42aa6ba admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.620844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cc1e1-c5a6-4245-a2db-6725f42aa6ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.622753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cc1e1-c5a6-4245-a2db-6725f42aa6ba admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.646035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1014/2036] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:38 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:38.650281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2500bc-c8d2-4240-9382-0772182c72ec admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.661271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2500bc-c8d2-4240-9382-0772182c72ec admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.666889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885f736-7bbd-4f0f-8420-eb50106a320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.667559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885f736-7bbd-4f0f-8420-eb50106a320e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.668193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885f736-7bbd-4f0f-8420-eb50106a320e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.668193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885f736-7bbd-4f0f-8420-eb50106a320e admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.668876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885f736-7bbd-4f0f-8420-eb50106a320e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.670491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885f736-7bbd-4f0f-8420-eb50106a320e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.681351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1023/2037] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:38.685496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cc1e1-c5a6-4245-a2db-6725f42aa6ba admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.696016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cc1e1-c5a6-4245-a2db-6725f42aa6ba admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.701415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced475d-23bb-42d7-b72b-7f446b041a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.702371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced475d-23bb-42d7-b72b-7f446b041a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.702668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced475d-23bb-42d7-b72b-7f446b041a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.702940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced475d-23bb-42d7-b72b-7f446b041a36 admin admin] PATH_INFO: `/v3/projects/3d337b2bb99f4ff4a68a81bc7180186f/users/cd9e947b6b0b44d6af42a383e99fb1d5/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.719210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced475d-23bb-42d7-b72b-7f446b041a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d337b2bb99f4ff4a68a81bc7180186f, user_id=cd9e947b6b0b44d6af42a383e99fb1d5, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.721924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced475d-23bb-42d7-b72b-7f446b041a36 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.736856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1015/2038] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:38 2022] PUT /identity/v3/projects/3d337b2bb99f4ff4a68a81bc7180186f/users/cd9e947b6b0b44d6af42a383e99fb1d5/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:38.741071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885f736-7bbd-4f0f-8420-eb50106a320e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.751273 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885f736-7bbd-4f0f-8420-eb50106a320e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.811332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95d954-503e-4541-a954-33e678ed0e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.812066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95d954-503e-4541-a954-33e678ed0e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.812377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95d954-503e-4541-a954-33e678ed0e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.812654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95d954-503e-4541-a954-33e678ed0e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.813492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95d954-503e-4541-a954-33e678ed0e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.815117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95d954-503e-4541-a954-33e678ed0e8a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.825393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1024/2039] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:38.829737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced475d-23bb-42d7-b72b-7f446b041a36 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:38.841082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced475d-23bb-42d7-b72b-7f446b041a36 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:38.846823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ffbb6-bf4c-4edf-92fd-3b7158fb6669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:38.848534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ffbb6-bf4c-4edf-92fd-3b7158fb6669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.848882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ffbb6-bf4c-4edf-92fd-3b7158fb6669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.849513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ffbb6-bf4c-4edf-92fd-3b7158fb6669 admin admin] PATH_INFO: `/v3/projects/3d337b2bb99f4ff4a68a81bc7180186f/users/cd9e947b6b0b44d6af42a383e99fb1d5/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.852157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ffbb6-bf4c-4edf-92fd-3b7158fb6669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d337b2bb99f4ff4a68a81bc7180186f, user_id=cd9e947b6b0b44d6af42a383e99fb1d5, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:38.855172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ffbb6-bf4c-4edf-92fd-3b7158fb6669 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:38.877879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1016/2040] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:38 2022] PUT /identity/v3/projects/3d337b2bb99f4ff4a68a81bc7180186f/users/cd9e947b6b0b44d6af42a383e99fb1d5/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:38.880256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad5cc8-edea-4bb7-8071-455f78c58177 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:38.880653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad5cc8-edea-4bb7-8071-455f78c58177 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:38.881173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad5cc8-edea-4bb7-8071-455f78c58177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:38.888837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:38.915934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-85ad5cc8-edea-4bb7-8071-455f78c58177 None None] MFA Rules not processed for user `cd9e947b6b0b44d6af42a383e99fb1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:38.917193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-85ad5cc8-edea-4bb7-8071-455f78c58177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:38.997727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1025/2041] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:39.009740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ffbb6-bf4c-4edf-92fd-3b7158fb6669 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:39.011525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-962ffbb6-bf4c-4edf-92fd-3b7158fb6669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:39.026363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:39.073272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ffbb6-bf4c-4edf-92fd-3b7158fb6669 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:39.077083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126163d-4a76-4b4b-95c6-1d4faffb7a2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:39.077746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126163d-4a76-4b4b-95c6-1d4faffb7a2d service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:39.078024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126163d-4a76-4b4b-95c6-1d4faffb7a2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:39.078295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126163d-4a76-4b4b-95c6-1d4faffb7a2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:39.143309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126163d-4a76-4b4b-95c6-1d4faffb7a2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:39.145080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126163d-4a76-4b4b-95c6-1d4faffb7a2d service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:39.157027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1017/2042] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:40.199602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad5cc8-edea-4bb7-8071-455f78c58177 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:40.201218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-85ad5cc8-edea-4bb7-8071-455f78c58177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:40.269913 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad5cc8-edea-4bb7-8071-455f78c58177 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:40.275354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:40.276362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:40.276773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:40.277046 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] PATH_INFO: `/v3/users/43a126f2963e46e3854595fc7de76ed9` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:40.278564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43a126f2963e46e3854595fc7de76ed9)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:40.281008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:40.335295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43a126f2963e46e3854595fc7de76ed9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:40.347298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43a126f2963e46e3854595fc7de76ed9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:40.357340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43a126f2963e46e3854595fc7de76ed9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:40.357966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43a126f2963e46e3854595fc7de76ed9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:40.365311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43a126f2963e46e3854595fc7de76ed9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:40.371605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1026/2043] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:40 2022] DELETE /identity/v3/users/43a126f2963e46e3854595fc7de76ed9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:40.376220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991a298-ffd1-43cb-8f4b-b47a84d44ac4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:40.376677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991a298-ffd1-43cb-8f4b-b47a84d44ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:40.376885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991a298-ffd1-43cb-8f4b-b47a84d44ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:40.394891 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2991a298-ffd1-43cb-8f4b-b47a84d44ac4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:40.396099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2991a298-ffd1-43cb-8f4b-b47a84d44ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:40.474687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1018/2044] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:40.487256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:40.488701 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:40.501095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:40.549912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9697f4b-80c5-4d82-bae8-77cd26b76571 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:40.554149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8048d7f-edcd-4cd9-94c3-bee490ff15bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:40.554805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8048d7f-edcd-4cd9-94c3-bee490ff15bd service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:40.555090 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8048d7f-edcd-4cd9-94c3-bee490ff15bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:40.555365 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8048d7f-edcd-4cd9-94c3-bee490ff15bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:40.566046 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8048d7f-edcd-4cd9-94c3-bee490ff15bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:40.567672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8048d7f-edcd-4cd9-94c3-bee490ff15bd service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:40.578221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1027/2045] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:40.894314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991a298-ffd1-43cb-8f4b-b47a84d44ac4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:40.897330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2991a298-ffd1-43cb-8f4b-b47a84d44ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:40.918065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991a298-ffd1-43cb-8f4b-b47a84d44ac4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:40.923806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:40.924818 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:40.925105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:40.925421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] PATH_INFO: `/v3/projects/efd6fb82be9f45fe933b71c7ae99f3ec` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:40.926634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd6fb82be9f45fe933b71c7ae99f3ec)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:40.928643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:40.979504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd6fb82be9f45fe933b71c7ae99f3ec', 'request_id': 'req-f6571bcd-66c5-4e65-b296-18aabe7f6786'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:40.985786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd6fb82be9f45fe933b71c7ae99f3ec', 'request_id': 'req-f6571bcd-66c5-4e65-b296-18aabe7f6786'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:40.995114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd6fb82be9f45fe933b71c7ae99f3ec', 'request_id': 'req-f6571bcd-66c5-4e65-b296-18aabe7f6786'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:40.997419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:41.009029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] The token cache is being invalidate because project efd6fb82be9f45fe933b71c7ae99f3ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:41.009544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.010586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.012183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1019/2046] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:40 2022] DELETE /identity/v3/projects/efd6fb82be9f45fe933b71c7ae99f3ec => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:41.016394 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8048d7f-edcd-4cd9-94c3-bee490ff15bd service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:41.031224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8048d7f-edcd-4cd9-94c3-bee490ff15bd service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:41.108110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:41.109063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:41.109349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:41.109628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] PATH_INFO: `/v3/users/830edfaff68c49f288320996a41b635c` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:41.111027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=830edfaff68c49f288320996a41b635c)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:41.117839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:41.186223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830edfaff68c49f288320996a41b635c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.199034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830edfaff68c49f288320996a41b635c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.208059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830edfaff68c49f288320996a41b635c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.208714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830edfaff68c49f288320996a41b635c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.216077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f7809cd7-abdf-4455-bab8-9305bd7320ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830edfaff68c49f288320996a41b635c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.222233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1028/2047] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:41 2022] DELETE /identity/v3/users/830edfaff68c49f288320996a41b635c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:41.518678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:41.521185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:41.535917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6571bcd-66c5-4e65-b296-18aabe7f6786 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:41.600799 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:41.601720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:41.602001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:41.602274 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] PATH_INFO: `/v3/projects/8aaa0b2e79be4e4893c81bf306fa0b10` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:41.603464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aaa0b2e79be4e4893c81bf306fa0b10)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:41.605432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:41.653987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aaa0b2e79be4e4893c81bf306fa0b10', 'request_id': 'req-a65644b2-096e-46c0-aea0-f2451cf63e1b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.660623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aaa0b2e79be4e4893c81bf306fa0b10', 'request_id': 'req-a65644b2-096e-46c0-aea0-f2451cf63e1b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.672106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aaa0b2e79be4e4893c81bf306fa0b10', 'request_id': 'req-a65644b2-096e-46c0-aea0-f2451cf63e1b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.684109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] The token cache is being invalidate because project 8aaa0b2e79be4e4893c81bf306fa0b10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:41.684399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.685075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a65644b2-096e-46c0-aea0-f2451cf63e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:41.686684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1020/2048] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:41 2022] DELETE /identity/v3/projects/8aaa0b2e79be4e4893c81bf306fa0b10 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:41.694082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad9d1-7849-4832-85f6-a4fc6bb0702b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:41.694393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad9d1-7849-4832-85f6-a4fc6bb0702b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:41.694680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad9d1-7849-4832-85f6-a4fc6bb0702b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:41.714590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-616ad9d1-7849-4832-85f6-a4fc6bb0702b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:41.716205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-616ad9d1-7849-4832-85f6-a4fc6bb0702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:41.745266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:41.809364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1029/2049] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:41.823527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90878c60-03b4-42f2-aab0-9ca0353c5feb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:41.823782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90878c60-03b4-42f2-aab0-9ca0353c5feb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:41.824181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90878c60-03b4-42f2-aab0-9ca0353c5feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:41.882882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-90878c60-03b4-42f2-aab0-9ca0353c5feb None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:41.884343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-90878c60-03b4-42f2-aab0-9ca0353c5feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:41.894172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1021/2050] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:41.898571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad9d1-7849-4832-85f6-a4fc6bb0702b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:41.909278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad9d1-7849-4832-85f6-a4fc6bb0702b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:41.914335 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a17c02-8f27-4cf1-94da-75448ba2576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:41.915132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a17c02-8f27-4cf1-94da-75448ba2576a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:41.915484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a17c02-8f27-4cf1-94da-75448ba2576a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:41.915828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a17c02-8f27-4cf1-94da-75448ba2576a admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:41.916641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a17c02-8f27-4cf1-94da-75448ba2576a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:41.917043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a17c02-8f27-4cf1-94da-75448ba2576a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:41.918728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a17c02-8f27-4cf1-94da-75448ba2576a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:41.931717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1030/2051] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:41.936918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90878c60-03b4-42f2-aab0-9ca0353c5feb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:41.949978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90878c60-03b4-42f2-aab0-9ca0353c5feb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:41.955317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2870efe-1a2d-48e9-9ab1-3db23798c959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:41.956035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2870efe-1a2d-48e9-9ab1-3db23798c959 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:41.956446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2870efe-1a2d-48e9-9ab1-3db23798c959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:41.956728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2870efe-1a2d-48e9-9ab1-3db23798c959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:41.957795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2870efe-1a2d-48e9-9ab1-3db23798c959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:41.959788 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2870efe-1a2d-48e9-9ab1-3db23798c959 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:41.991339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1022/2052] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:41 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:41.995634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a17c02-8f27-4cf1-94da-75448ba2576a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:42.006052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a17c02-8f27-4cf1-94da-75448ba2576a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:42.064762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873fbce-eac7-483f-8e8a-ce7315f85f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:42.065426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873fbce-eac7-483f-8e8a-ce7315f85f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:42.065714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873fbce-eac7-483f-8e8a-ce7315f85f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:42.065981 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873fbce-eac7-483f-8e8a-ce7315f85f98 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:42.066768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873fbce-eac7-483f-8e8a-ce7315f85f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:42.068697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873fbce-eac7-483f-8e8a-ce7315f85f98 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:42.091479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1031/2053] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:41 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:42.095853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2870efe-1a2d-48e9-9ab1-3db23798c959 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:42.106603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2870efe-1a2d-48e9-9ab1-3db23798c959 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:42.111554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77f4b1-bf2b-4381-ae78-63e0d18b09ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:42.112217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77f4b1-bf2b-4381-ae78-63e0d18b09ac admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:42.112500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77f4b1-bf2b-4381-ae78-63e0d18b09ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:42.112801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77f4b1-bf2b-4381-ae78-63e0d18b09ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:42.113640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77f4b1-bf2b-4381-ae78-63e0d18b09ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:42.115241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77f4b1-bf2b-4381-ae78-63e0d18b09ac admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:42.126226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1023/2054] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:42.130374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873fbce-eac7-483f-8e8a-ce7315f85f98 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:42.140697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873fbce-eac7-483f-8e8a-ce7315f85f98 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:42.145848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d6b3e-fb03-4e38-90fe-14d8fe1ce1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:42.146504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d6b3e-fb03-4e38-90fe-14d8fe1ce1ab admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:42.146773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d6b3e-fb03-4e38-90fe-14d8fe1ce1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:42.147048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d6b3e-fb03-4e38-90fe-14d8fe1ce1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:42.147778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d6b3e-fb03-4e38-90fe-14d8fe1ce1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:42.149414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d6b3e-fb03-4e38-90fe-14d8fe1ce1ab admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:42.159958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1032/2055] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:42.164237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77f4b1-bf2b-4381-ae78-63e0d18b09ac admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:42.174312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77f4b1-bf2b-4381-ae78-63e0d18b09ac admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:42.179121 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895227a7-2bca-4062-bd72-e7c3f7cda70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:42.180017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895227a7-2bca-4062-bd72-e7c3f7cda70d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:42.180299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895227a7-2bca-4062-bd72-e7c3f7cda70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:42.180567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895227a7-2bca-4062-bd72-e7c3f7cda70d admin admin] PATH_INFO: `/v3/projects/68207b0d0b3c48ceabc7828d0db8825f/users/96f5680d4d4c4c34b8e8759efaf64515/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:42.198423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895227a7-2bca-4062-bd72-e7c3f7cda70d admin admin] RBAC: Authorizing `identity:create_grant(project_id=68207b0d0b3c48ceabc7828d0db8825f, user_id=96f5680d4d4c4c34b8e8759efaf64515, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:42.201311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895227a7-2bca-4062-bd72-e7c3f7cda70d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:42.232763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1024/2056] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:42 2022] PUT /identity/v3/projects/68207b0d0b3c48ceabc7828d0db8825f/users/96f5680d4d4c4c34b8e8759efaf64515/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:42.233298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7a91e-be7b-4d5b-b7b2-f1cf913e6025 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:42.233298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7a91e-be7b-4d5b-b7b2-f1cf913e6025 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:42.233298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7a91e-be7b-4d5b-b7b2-f1cf913e6025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:42.263717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-48f7a91e-be7b-4d5b-b7b2-f1cf913e6025 None None] MFA Rules not processed for user `96f5680d4d4c4c34b8e8759efaf64515`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:42.265065 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-48f7a91e-be7b-4d5b-b7b2-f1cf913e6025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:42.336769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1033/2057] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:42.347639 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895227a7-2bca-4062-bd72-e7c3f7cda70d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:42.349709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-895227a7-2bca-4062-bd72-e7c3f7cda70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:42.388255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:42.453061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895227a7-2bca-4062-bd72-e7c3f7cda70d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:42.457950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bd0f7-7316-4a84-b45a-e032369648f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b75a27126c9464989cec70e5104824c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:42.458749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bd0f7-7316-4a84-b45a-e032369648f5 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:42.459040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bd0f7-7316-4a84-b45a-e032369648f5 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:42.459355 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bd0f7-7316-4a84-b45a-e032369648f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:42.471110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bd0f7-7316-4a84-b45a-e032369648f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:42.472581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bd0f7-7316-4a84-b45a-e032369648f5 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:42.483288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1025/2058] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:43.810199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7a91e-be7b-4d5b-b7b2-f1cf913e6025 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:43.813532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-48f7a91e-be7b-4d5b-b7b2-f1cf913e6025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:43.850051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:43.908983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7a91e-be7b-4d5b-b7b2-f1cf913e6025 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:43.917383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5db2f-ccc1-41cc-9d5e-2930fe5d78a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:43.918158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5db2f-ccc1-41cc-9d5e-2930fe5d78a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:43.918514 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5db2f-ccc1-41cc-9d5e-2930fe5d78a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:43.918854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5db2f-ccc1-41cc-9d5e-2930fe5d78a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:43.921956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1ea5db2f-ccc1-41cc-9d5e-2930fe5d78a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:44.044252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5db2f-ccc1-41cc-9d5e-2930fe5d78a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:44.047403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5db2f-ccc1-41cc-9d5e-2930fe5d78a6 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:44.058849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1034/2059] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:45.378231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bd0f7-7316-4a84-b45a-e032369648f5 service glance] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:45.380781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d04bd0f7-7316-4a84-b45a-e032369648f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:45.462243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bd0f7-7316-4a84-b45a-e032369648f5 service glance] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:45.467010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:45.468281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:45.468424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:45.468613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] PATH_INFO: `/v3/users/3ade449bab27445693224e6bf1e881ac` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:45.469832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ade449bab27445693224e6bf1e881ac)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:45.471758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:45.541216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ade449bab27445693224e6bf1e881ac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:45.563946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ade449bab27445693224e6bf1e881ac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:45.575508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ade449bab27445693224e6bf1e881ac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:45.605385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ade449bab27445693224e6bf1e881ac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:45.606896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ade449bab27445693224e6bf1e881ac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:45.616105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1026/2060] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:45 2022] DELETE /identity/v3/users/3ade449bab27445693224e6bf1e881ac => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:45.620574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5a1d1-49c2-4827-8e26-cfca0874134f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:45.621464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5a1d1-49c2-4827-8e26-cfca0874134f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:45.621751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5a1d1-49c2-4827-8e26-cfca0874134f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:45.645482 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-80a5a1d1-49c2-4827-8e26-cfca0874134f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:45.647353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-80a5a1d1-49c2-4827-8e26-cfca0874134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:45.729756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1035/2061] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:45.740791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:45.742322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:45.800194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:45.823853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bb834-cde8-4c45-abd1-eed9731cdde8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:45.833350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac727c8-3031-4fee-b1f6-23e54c801f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:45.834177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac727c8-3031-4fee-b1f6-23e54c801f59 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:45.834559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac727c8-3031-4fee-b1f6-23e54c801f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:45.834992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac727c8-3031-4fee-b1f6-23e54c801f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:45.845563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac727c8-3031-4fee-b1f6-23e54c801f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:45.847121 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac727c8-3031-4fee-b1f6-23e54c801f59 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:45.859881 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1027/2062] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:46.169146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5a1d1-49c2-4827-8e26-cfca0874134f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:46.170415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-80a5a1d1-49c2-4827-8e26-cfca0874134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:46.184778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5a1d1-49c2-4827-8e26-cfca0874134f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:46.190465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:46.192141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:46.192513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:46.193069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] PATH_INFO: `/v3/projects/ee47f7edd88042a3a9508e3e24343c86` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:46.194606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee47f7edd88042a3a9508e3e24343c86)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:46.197389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:46.257466 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee47f7edd88042a3a9508e3e24343c86', 'request_id': 'req-7ce06372-2f28-4725-9d5d-c478119ddc77'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:46.266403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee47f7edd88042a3a9508e3e24343c86', 'request_id': 'req-7ce06372-2f28-4725-9d5d-c478119ddc77'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:46.274286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee47f7edd88042a3a9508e3e24343c86', 'request_id': 'req-7ce06372-2f28-4725-9d5d-c478119ddc77'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:46.283244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:46.292785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] The token cache is being invalidate because project ee47f7edd88042a3a9508e3e24343c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:46.293082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:46.293761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce06372-2f28-4725-9d5d-c478119ddc77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:46.295621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1036/2063] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:46 2022] DELETE /identity/v3/projects/ee47f7edd88042a3a9508e3e24343c86 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:49.150522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac727c8-3031-4fee-b1f6-23e54c801f59 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:49.152016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9ac727c8-3031-4fee-b1f6-23e54c801f59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:49.165045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac727c8-3031-4fee-b1f6-23e54c801f59 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:49.227015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:49.227915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:49.228194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:49.228467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] PATH_INFO: `/v3/users/e11357bda6cb4b7a9cd59836e831c353` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:49.229697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11357bda6cb4b7a9cd59836e831c353)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:49.231641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:49.295482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11357bda6cb4b7a9cd59836e831c353'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:49.318740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11357bda6cb4b7a9cd59836e831c353'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:49.328708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11357bda6cb4b7a9cd59836e831c353'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:49.337724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11357bda6cb4b7a9cd59836e831c353'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:49.338453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11357bda6cb4b7a9cd59836e831c353'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:49.348829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1028/2064] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:49 2022] DELETE /identity/v3/users/e11357bda6cb4b7a9cd59836e831c353 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:49.351317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336e33a-b0b9-4b33-bbc0-9b7ddc81b553 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:49.351834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336e33a-b0b9-4b33-bbc0-9b7ddc81b553 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:49.352426 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336e33a-b0b9-4b33-bbc0-9b7ddc81b553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:49.370868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0336e33a-b0b9-4b33-bbc0-9b7ddc81b553 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:49.372425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0336e33a-b0b9-4b33-bbc0-9b7ddc81b553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:49.449210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1037/2065] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:49.459960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:49.461782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:49.540280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f05547-8edb-49e6-ae07-874afd62bc45 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:49.543945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ac583-1163-42be-be9f-3ca8956f430c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:49.544620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ac583-1163-42be-be9f-3ca8956f430c service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:49.544936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ac583-1163-42be-be9f-3ca8956f430c service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:49.545216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ac583-1163-42be-be9f-3ca8956f430c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:49.561318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ac583-1163-42be-be9f-3ca8956f430c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:49.563935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ac583-1163-42be-be9f-3ca8956f430c service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:49.576393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1029/2066] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:50.043928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336e33a-b0b9-4b33-bbc0-9b7ddc81b553 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:50.045688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0336e33a-b0b9-4b33-bbc0-9b7ddc81b553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:50.060404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336e33a-b0b9-4b33-bbc0-9b7ddc81b553 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:50.065428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:50.066368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:50.066821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:50.067111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] PATH_INFO: `/v3/projects/5c004f9c67094294be15e2cbc4f6db2e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:50.068318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c004f9c67094294be15e2cbc4f6db2e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:50.072014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:50.144944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c004f9c67094294be15e2cbc4f6db2e', 'request_id': 'req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.154675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c004f9c67094294be15e2cbc4f6db2e', 'request_id': 'req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.163921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c004f9c67094294be15e2cbc4f6db2e', 'request_id': 'req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.175500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] The token cache is being invalidate because project 5c004f9c67094294be15e2cbc4f6db2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:50.175856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.176786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.178354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1038/2067] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:50 2022] DELETE /identity/v3/projects/5c004f9c67094294be15e2cbc4f6db2e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:50.182528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ac583-1163-42be-be9f-3ca8956f430c service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:50.188565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:50.193155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ac583-1163-42be-be9f-3ca8956f430c service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:50.284485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:50.285624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:50.286048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:50.286357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] PATH_INFO: `/v3/users/cd9e947b6b0b44d6af42a383e99fb1d5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:50.287728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd9e947b6b0b44d6af42a383e99fb1d5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:50.289964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:50.360662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd9e947b6b0b44d6af42a383e99fb1d5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.377594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd9e947b6b0b44d6af42a383e99fb1d5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.391037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd9e947b6b0b44d6af42a383e99fb1d5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.401279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd9e947b6b0b44d6af42a383e99fb1d5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.401908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-be9e36b5-f539-408d-a226-ae50862a0465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd9e947b6b0b44d6af42a383e99fb1d5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:50.409194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1030/2068] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:50 2022] DELETE /identity/v3/users/cd9e947b6b0b44d6af42a383e99fb1d5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:51.099845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:51.101386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:51.123751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff66d5-e97d-4ef1-9624-64a6f6452ec3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:51.197115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:51.198059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.198344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.198623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] PATH_INFO: `/v3/projects/3d337b2bb99f4ff4a68a81bc7180186f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.200000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d337b2bb99f4ff4a68a81bc7180186f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:51.202004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:51.235449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:51.285580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d337b2bb99f4ff4a68a81bc7180186f', 'request_id': 'req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:51.296071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d337b2bb99f4ff4a68a81bc7180186f', 'request_id': 'req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:51.304565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d337b2bb99f4ff4a68a81bc7180186f', 'request_id': 'req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:51.319060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] The token cache is being invalidate because project 3d337b2bb99f4ff4a68a81bc7180186f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:51.319486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:51.320175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c6c27526-b521-4cbc-afe6-8c74b1e1e2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:51.326433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1039/2069] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:51 2022] DELETE /identity/v3/projects/3d337b2bb99f4ff4a68a81bc7180186f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:51.328354 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc145548-7335-4129-b7d9-108b744f478f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.328887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc145548-7335-4129-b7d9-108b744f478f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.329245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc145548-7335-4129-b7d9-108b744f478f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.356095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cc145548-7335-4129-b7d9-108b744f478f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:51.357466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cc145548-7335-4129-b7d9-108b744f478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:51.438201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1031/2070] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:51.448671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724dd1-9ea5-4ca3-b0b9-1490783c54e5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.448974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724dd1-9ea5-4ca3-b0b9-1490783c54e5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.449317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724dd1-9ea5-4ca3-b0b9-1490783c54e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.474592 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fc724dd1-9ea5-4ca3-b0b9-1490783c54e5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:51.476276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fc724dd1-9ea5-4ca3-b0b9-1490783c54e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:51.487249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1040/2071] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:51.492612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc145548-7335-4129-b7d9-108b744f478f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:51.503080 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc145548-7335-4129-b7d9-108b744f478f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:51.509264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80287c1c-a4a8-4810-84e2-09f80451dce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:51.510059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80287c1c-a4a8-4810-84e2-09f80451dce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.510473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80287c1c-a4a8-4810-84e2-09f80451dce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.511203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80287c1c-a4a8-4810-84e2-09f80451dce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.512041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80287c1c-a4a8-4810-84e2-09f80451dce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:51.512453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80287c1c-a4a8-4810-84e2-09f80451dce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:51.514649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80287c1c-a4a8-4810-84e2-09f80451dce7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:51.533567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1032/2072] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:51.539248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc724dd1-9ea5-4ca3-b0b9-1490783c54e5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:51.550790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc724dd1-9ea5-4ca3-b0b9-1490783c54e5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:51.556270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd8f21-241c-40d1-9798-de969f3db010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:51.556661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd8f21-241c-40d1-9798-de969f3db010 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.557016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd8f21-241c-40d1-9798-de969f3db010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.557442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd8f21-241c-40d1-9798-de969f3db010 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.558597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd8f21-241c-40d1-9798-de969f3db010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:51.560899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd8f21-241c-40d1-9798-de969f3db010 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:51.599267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1041/2073] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:51 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:51.601876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80287c1c-a4a8-4810-84e2-09f80451dce7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:51.618912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80287c1c-a4a8-4810-84e2-09f80451dce7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:51.630438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:51.712894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bdea3-9316-4751-a8b1-c98318a63880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:51.713499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bdea3-9316-4751-a8b1-c98318a63880 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.713781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bdea3-9316-4751-a8b1-c98318a63880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.714099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bdea3-9316-4751-a8b1-c98318a63880 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.714756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bdea3-9316-4751-a8b1-c98318a63880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:51.717202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bdea3-9316-4751-a8b1-c98318a63880 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:51.744002 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1033/2074] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:51 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:51.748342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd8f21-241c-40d1-9798-de969f3db010 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:51.760420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd8f21-241c-40d1-9798-de969f3db010 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:51.765608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fcb49-9cc7-4cce-858d-f1ca323d8f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:51.766288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fcb49-9cc7-4cce-858d-f1ca323d8f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.766808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fcb49-9cc7-4cce-858d-f1ca323d8f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.767095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fcb49-9cc7-4cce-858d-f1ca323d8f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.767841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fcb49-9cc7-4cce-858d-f1ca323d8f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:51.770002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fcb49-9cc7-4cce-858d-f1ca323d8f32 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:51.786921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1042/2075] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:51.791392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bdea3-9316-4751-a8b1-c98318a63880 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:51.808086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bdea3-9316-4751-a8b1-c98318a63880 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:51.813741 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc867-9812-4443-8580-bf0dc7f3e78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:51.814956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc867-9812-4443-8580-bf0dc7f3e78a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.815418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc867-9812-4443-8580-bf0dc7f3e78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.815840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc867-9812-4443-8580-bf0dc7f3e78a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.817483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bc867-9812-4443-8580-bf0dc7f3e78a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:51.819938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bc867-9812-4443-8580-bf0dc7f3e78a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:51.831247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1034/2076] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:51.836429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fcb49-9cc7-4cce-858d-f1ca323d8f32 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:51.847796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fcb49-9cc7-4cce-858d-f1ca323d8f32 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:51.853176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be0d80-3ea2-4200-a909-121c92ded8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:51.854089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be0d80-3ea2-4200-a909-121c92ded8a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.854448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be0d80-3ea2-4200-a909-121c92ded8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.854633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be0d80-3ea2-4200-a909-121c92ded8a0 admin admin] PATH_INFO: `/v3/projects/13b14c31a0d34343b8ed80a96bad98f5/users/e59e20e621424abd99d096c37cccaa21/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.900075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be0d80-3ea2-4200-a909-121c92ded8a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b14c31a0d34343b8ed80a96bad98f5, user_id=e59e20e621424abd99d096c37cccaa21, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:51.906183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be0d80-3ea2-4200-a909-121c92ded8a0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:51.915875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc867-9812-4443-8580-bf0dc7f3e78a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:51.918316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4d1bc867-9812-4443-8580-bf0dc7f3e78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:51.937438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1043/2077] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:51 2022] PUT /identity/v3/projects/13b14c31a0d34343b8ed80a96bad98f5/users/e59e20e621424abd99d096c37cccaa21/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:51.940344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc867-9812-4443-8580-bf0dc7f3e78a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:51.946814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8a27-944a-4a06-8799-2aca0c1d0300 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:51.947107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8a27-944a-4a06-8799-2aca0c1d0300 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:51.947386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8a27-944a-4a06-8799-2aca0c1d0300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:51.975918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9dcc8a27-944a-4a06-8799-2aca0c1d0300 None None] MFA Rules not processed for user `e59e20e621424abd99d096c37cccaa21`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:51.977285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9dcc8a27-944a-4a06-8799-2aca0c1d0300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:52.009768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:52.010666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.010961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.011227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] PATH_INFO: `/v3/users/16daeab4accb4eeca4100b55dc9bcaa2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.012922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16daeab4accb4eeca4100b55dc9bcaa2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:52.015248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:52.060152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1044/2078] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:52.074549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8a27-944a-4a06-8799-2aca0c1d0300 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:52.075976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9dcc8a27-944a-4a06-8799-2aca0c1d0300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:52.078989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16daeab4accb4eeca4100b55dc9bcaa2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.095095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16daeab4accb4eeca4100b55dc9bcaa2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.115702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16daeab4accb4eeca4100b55dc9bcaa2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.122587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:52.123832 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16daeab4accb4eeca4100b55dc9bcaa2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.124565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16daeab4accb4eeca4100b55dc9bcaa2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.132082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1035/2079] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:51 2022] DELETE /identity/v3/users/16daeab4accb4eeca4100b55dc9bcaa2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:52.140087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8a27-944a-4a06-8799-2aca0c1d0300 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:52.196167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9cac8-c65e-42fa-bd96-fec3ba8afe40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:52.196862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9cac8-c65e-42fa-bd96-fec3ba8afe40 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.197155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9cac8-c65e-42fa-bd96-fec3ba8afe40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.197438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9cac8-c65e-42fa-bd96-fec3ba8afe40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.201153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5ec9cac8-c65e-42fa-bd96-fec3ba8afe40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:52.263636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9cac8-c65e-42fa-bd96-fec3ba8afe40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:52.265132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9cac8-c65e-42fa-bd96-fec3ba8afe40 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:52.275187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1045/2080] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:36:52.318192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:52.319929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:52.334338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:52.398143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ac079-1ed2-47e7-b7e7-244611729815 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:52.405402 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:52.406290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.406578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.406853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] PATH_INFO: `/v3/projects/dee3d0adc7084828b9ed954edee54f6a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.408065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee3d0adc7084828b9ed954edee54f6a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:52.410441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:52.473821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee3d0adc7084828b9ed954edee54f6a', 'request_id': 'req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.482765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee3d0adc7084828b9ed954edee54f6a', 'request_id': 'req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.491220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee3d0adc7084828b9ed954edee54f6a', 'request_id': 'req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.504137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] The token cache is being invalidate because project dee3d0adc7084828b9ed954edee54f6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:52.504880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.505587 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b0f5f47f-e911-4a7f-ad5c-ee461aaab3a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:52.507246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1036/2081] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:52 2022] DELETE /identity/v3/projects/dee3d0adc7084828b9ed954edee54f6a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:52.512774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f6873-465f-44d0-be03-645f61967e54 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.513100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f6873-465f-44d0-be03-645f61967e54 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.513340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f6873-465f-44d0-be03-645f61967e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.536526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-665f6873-465f-44d0-be03-645f61967e54 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:52.537840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-665f6873-465f-44d0-be03-645f61967e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:52.631749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1046/2082] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:36:52.640014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22757439-187e-4e0f-b287-65f6abe0990d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.640623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22757439-187e-4e0f-b287-65f6abe0990d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.640960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22757439-187e-4e0f-b287-65f6abe0990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.670398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-22757439-187e-4e0f-b287-65f6abe0990d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:52.671750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-22757439-187e-4e0f-b287-65f6abe0990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:52.682222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1037/2083] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:52.686926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f6873-465f-44d0-be03-645f61967e54 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:52.698459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f6873-465f-44d0-be03-645f61967e54 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:52.704781 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f262e2-0a9d-472b-be0f-9cccc32e7462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:52.705268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f262e2-0a9d-472b-be0f-9cccc32e7462 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.705554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f262e2-0a9d-472b-be0f-9cccc32e7462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.705836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f262e2-0a9d-472b-be0f-9cccc32e7462 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.706577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f262e2-0a9d-472b-be0f-9cccc32e7462 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:52.706885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f262e2-0a9d-472b-be0f-9cccc32e7462 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:52.708699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f262e2-0a9d-472b-be0f-9cccc32e7462 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:52.721266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1047/2084] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:52.725761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22757439-187e-4e0f-b287-65f6abe0990d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:52.735430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22757439-187e-4e0f-b287-65f6abe0990d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:52.740198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f925f8-0faf-4486-adb1-e748a4b872cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:52.740837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f925f8-0faf-4486-adb1-e748a4b872cf admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.741122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f925f8-0faf-4486-adb1-e748a4b872cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.741398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f925f8-0faf-4486-adb1-e748a4b872cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.742108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f925f8-0faf-4486-adb1-e748a4b872cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:52.744036 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f925f8-0faf-4486-adb1-e748a4b872cf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:52.775163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1038/2085] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:52 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:36:52.776473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f262e2-0a9d-472b-be0f-9cccc32e7462 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:52.787835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f262e2-0a9d-472b-be0f-9cccc32e7462 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:52.824812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:52.863034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326af84-43ee-43a1-912c-671d6d48c05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:52.865450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326af84-43ee-43a1-912c-671d6d48c05e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.865746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326af84-43ee-43a1-912c-671d6d48c05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.866035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326af84-43ee-43a1-912c-671d6d48c05e admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.866742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326af84-43ee-43a1-912c-671d6d48c05e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:52.868691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326af84-43ee-43a1-912c-671d6d48c05e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:52.890344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1048/2086] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:52 2022] POST /identity/v3/users => generated 282 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:52.894482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f925f8-0faf-4486-adb1-e748a4b872cf admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:52.906650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f925f8-0faf-4486-adb1-e748a4b872cf admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:52.912401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3603c9d-9324-4707-8aa3-0c785f9612d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:52.913151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3603c9d-9324-4707-8aa3-0c785f9612d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.913475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3603c9d-9324-4707-8aa3-0c785f9612d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.913779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3603c9d-9324-4707-8aa3-0c785f9612d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.914606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3603c9d-9324-4707-8aa3-0c785f9612d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:52.916360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3603c9d-9324-4707-8aa3-0c785f9612d3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:52.933098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1039/2087] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:52.936936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326af84-43ee-43a1-912c-671d6d48c05e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:52.950686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326af84-43ee-43a1-912c-671d6d48c05e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:52.955819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e154d-75a9-4857-988e-55b1984fc066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:52.956628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e154d-75a9-4857-988e-55b1984fc066 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.957091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e154d-75a9-4857-988e-55b1984fc066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.957374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e154d-75a9-4857-988e-55b1984fc066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:52.958222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e154d-75a9-4857-988e-55b1984fc066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:52.959799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e154d-75a9-4857-988e-55b1984fc066 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:52.974188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1049/2088] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:52.978032 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3603c9d-9324-4707-8aa3-0c785f9612d3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:52.989812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3603c9d-9324-4707-8aa3-0c785f9612d3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:52.994596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e7fdc-57e0-4273-ab7b-d445e4025a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:52.995479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e7fdc-57e0-4273-ab7b-d445e4025a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:52.995753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e7fdc-57e0-4273-ab7b-d445e4025a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:52.996047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e7fdc-57e0-4273-ab7b-d445e4025a6b admin admin] PATH_INFO: `/v3/projects/c137f78bb5be45d2b9e52c5e8c63d060/users/75f8e5b7e5b9491abdd428d0a76fd0e3/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.011047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e7fdc-57e0-4273-ab7b-d445e4025a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c137f78bb5be45d2b9e52c5e8c63d060, user_id=75f8e5b7e5b9491abdd428d0a76fd0e3, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:53.013770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e7fdc-57e0-4273-ab7b-d445e4025a6b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:53.029310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1040/2089] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:52 2022] PUT /identity/v3/projects/c137f78bb5be45d2b9e52c5e8c63d060/users/75f8e5b7e5b9491abdd428d0a76fd0e3/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:53.039506 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65027f3-e222-444b-88e6-b79e06f8fba7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:53.039824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65027f3-e222-444b-88e6-b79e06f8fba7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:53.040138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65027f3-e222-444b-88e6-b79e06f8fba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.068297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a65027f3-e222-444b-88e6-b79e06f8fba7 None None] MFA Rules not processed for user `75f8e5b7e5b9491abdd428d0a76fd0e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:53.069559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a65027f3-e222-444b-88e6-b79e06f8fba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:53.155861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1050/2090] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:53.160474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e7fdc-57e0-4273-ab7b-d445e4025a6b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:53.209445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:53.235792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e7fdc-57e0-4273-ab7b-d445e4025a6b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:53.242018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df51b3-c656-4f35-a639-ba91d15f21f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:53.242696 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df51b3-c656-4f35-a639-ba91d15f21f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:53.242984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df51b3-c656-4f35-a639-ba91d15f21f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:53.243257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df51b3-c656-4f35-a639-ba91d15f21f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.243994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df51b3-c656-4f35-a639-ba91d15f21f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:53.246323 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df51b3-c656-4f35-a639-ba91d15f21f4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:53.279466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1041/2091] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:53 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:53.283841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65027f3-e222-444b-88e6-b79e06f8fba7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:53.296068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65027f3-e222-444b-88e6-b79e06f8fba7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:53.358434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ce9e3-cb9d-45ff-a1c8-f61533da5b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:53.359119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ce9e3-cb9d-45ff-a1c8-f61533da5b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:53.359402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ce9e3-cb9d-45ff-a1c8-f61533da5b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:53.359680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ce9e3-cb9d-45ff-a1c8-f61533da5b80 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.360405 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ce9e3-cb9d-45ff-a1c8-f61533da5b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:53.362403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ce9e3-cb9d-45ff-a1c8-f61533da5b80 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:53.385772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1051/2092] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:53 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:53.389613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df51b3-c656-4f35-a639-ba91d15f21f4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:53.400335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df51b3-c656-4f35-a639-ba91d15f21f4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:53.405945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfeebd-451d-42f1-ae72-db181efce47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:53.406732 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfeebd-451d-42f1-ae72-db181efce47c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:53.407016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfeebd-451d-42f1-ae72-db181efce47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:53.407290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfeebd-451d-42f1-ae72-db181efce47c admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.408029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfeebd-451d-42f1-ae72-db181efce47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:53.409688 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfeebd-451d-42f1-ae72-db181efce47c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:53.421608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1042/2093] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:53.425645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ce9e3-cb9d-45ff-a1c8-f61533da5b80 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:53.440903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ce9e3-cb9d-45ff-a1c8-f61533da5b80 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:53.446157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34292c-df63-48df-8d48-aaeeb2f333b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:53.447153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34292c-df63-48df-8d48-aaeeb2f333b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:53.447511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34292c-df63-48df-8d48-aaeeb2f333b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:53.447865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34292c-df63-48df-8d48-aaeeb2f333b7 admin admin] PATH_INFO: `/v3/projects/0de36a237cf64be5a5188fe9646a7b7e/users/53bf8089e0c1405785ac37ab7fe4ff43/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.473504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34292c-df63-48df-8d48-aaeeb2f333b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de36a237cf64be5a5188fe9646a7b7e, user_id=53bf8089e0c1405785ac37ab7fe4ff43, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:53.476624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34292c-df63-48df-8d48-aaeeb2f333b7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:53.496698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1052/2094] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:53 2022] PUT /identity/v3/projects/0de36a237cf64be5a5188fe9646a7b7e/users/53bf8089e0c1405785ac37ab7fe4ff43/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:53.500691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfeebd-451d-42f1-ae72-db181efce47c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:53.511510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfeebd-451d-42f1-ae72-db181efce47c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:53.576887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128985d1-4aab-49f9-a2f4-3d760a450f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:53.577926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128985d1-4aab-49f9-a2f4-3d760a450f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:53.578209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128985d1-4aab-49f9-a2f4-3d760a450f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:53.578485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128985d1-4aab-49f9-a2f4-3d760a450f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.579229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128985d1-4aab-49f9-a2f4-3d760a450f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:53.580866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128985d1-4aab-49f9-a2f4-3d760a450f8c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:53.591644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1043/2095] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:53.595900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34292c-df63-48df-8d48-aaeeb2f333b7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:53.607295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34292c-df63-48df-8d48-aaeeb2f333b7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:53.612523 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de261a84-a8cd-4eaf-b040-db9e975e095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:53.613474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de261a84-a8cd-4eaf-b040-db9e975e095e admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:53.613790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de261a84-a8cd-4eaf-b040-db9e975e095e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:53.614114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de261a84-a8cd-4eaf-b040-db9e975e095e admin admin] PATH_INFO: `/v3/projects/0de36a237cf64be5a5188fe9646a7b7e/users/53bf8089e0c1405785ac37ab7fe4ff43/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.616358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de261a84-a8cd-4eaf-b040-db9e975e095e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de36a237cf64be5a5188fe9646a7b7e, user_id=53bf8089e0c1405785ac37ab7fe4ff43, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:53.620727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de261a84-a8cd-4eaf-b040-db9e975e095e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:53.625032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:53.636147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1053/2096] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:53 2022] PUT /identity/v3/projects/0de36a237cf64be5a5188fe9646a7b7e/users/53bf8089e0c1405785ac37ab7fe4ff43/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:53.648285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794047b8-47b4-41a0-88b4-e295fd758338 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:53.648592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794047b8-47b4-41a0-88b4-e295fd758338 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:53.648897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794047b8-47b4-41a0-88b4-e295fd758338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.679536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-794047b8-47b4-41a0-88b4-e295fd758338 None None] MFA Rules not processed for user `53bf8089e0c1405785ac37ab7fe4ff43`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:53.680774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-794047b8-47b4-41a0-88b4-e295fd758338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:53.760338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1044/2097] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:53.774463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de261a84-a8cd-4eaf-b040-db9e975e095e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:53.777008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-de261a84-a8cd-4eaf-b040-db9e975e095e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:53.848850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de261a84-a8cd-4eaf-b040-db9e975e095e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:53.853307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ca037-4daf-469d-b0f8-39bb5b5511f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:53.853958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ca037-4daf-469d-b0f8-39bb5b5511f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:53.854512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ca037-4daf-469d-b0f8-39bb5b5511f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:53.854512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ca037-4daf-469d-b0f8-39bb5b5511f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:53.878141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ca037-4daf-469d-b0f8-39bb5b5511f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:53.880345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ca037-4daf-469d-b0f8-39bb5b5511f1 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:53.890391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1054/2098] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:54.337058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794047b8-47b4-41a0-88b4-e295fd758338 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:54.372711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:54.404712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794047b8-47b4-41a0-88b4-e295fd758338 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:54.411934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:54.412884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:54.413163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:54.413423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] PATH_INFO: `/v3/users/75f8e5b7e5b9491abdd428d0a76fd0e3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:54.414668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f8e5b7e5b9491abdd428d0a76fd0e3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:54.416601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:54.470665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f8e5b7e5b9491abdd428d0a76fd0e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:54.484911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f8e5b7e5b9491abdd428d0a76fd0e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:54.504804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f8e5b7e5b9491abdd428d0a76fd0e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:54.512980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f8e5b7e5b9491abdd428d0a76fd0e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:54.513684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f8e5b7e5b9491abdd428d0a76fd0e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:54.520992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1045/2099] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:54 2022] DELETE /identity/v3/users/75f8e5b7e5b9491abdd428d0a76fd0e3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:54.525637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f6541-97ba-4caa-a781-26178d293953 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:54.526145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f6541-97ba-4caa-a781-26178d293953 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:54.526145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f6541-97ba-4caa-a781-26178d293953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:54.544723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2b0f6541-97ba-4caa-a781-26178d293953 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:54.545961 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2b0f6541-97ba-4caa-a781-26178d293953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:54.611057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1055/2100] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:54.621915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:54.623455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:54.687115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53762d8-de63-4c11-931c-9390ff8bd64a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:54.691284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7fae4-0059-4207-9633-3bc7ef0294f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:54.692242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7fae4-0059-4207-9633-3bc7ef0294f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:54.692629 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7fae4-0059-4207-9633-3bc7ef0294f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:54.692943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7fae4-0059-4207-9633-3bc7ef0294f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:54.706899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7fae4-0059-4207-9633-3bc7ef0294f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:54.708445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7fae4-0059-4207-9633-3bc7ef0294f5 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:54.718808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1046/2101] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:55.104584 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f6541-97ba-4caa-a781-26178d293953 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:55.106053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2b0f6541-97ba-4caa-a781-26178d293953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:55.118529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f6541-97ba-4caa-a781-26178d293953 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:55.123615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:55.124337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:55.124831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:55.125344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] PATH_INFO: `/v3/projects/c137f78bb5be45d2b9e52c5e8c63d060` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:55.126238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c137f78bb5be45d2b9e52c5e8c63d060)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:55.128184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:55.204010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c137f78bb5be45d2b9e52c5e8c63d060', 'request_id': 'req-5993742e-0726-4645-b9a1-a9d31b3a15fb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.213201 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c137f78bb5be45d2b9e52c5e8c63d060', 'request_id': 'req-5993742e-0726-4645-b9a1-a9d31b3a15fb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.220093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c137f78bb5be45d2b9e52c5e8c63d060', 'request_id': 'req-5993742e-0726-4645-b9a1-a9d31b3a15fb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.232473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] The token cache is being invalidate because project c137f78bb5be45d2b9e52c5e8c63d060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:55.232779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.233448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.234987 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1056/2102] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:55 2022] DELETE /identity/v3/projects/c137f78bb5be45d2b9e52c5e8c63d060 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:55.239157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7fae4-0059-4207-9633-3bc7ef0294f5 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:55.249078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7fae4-0059-4207-9633-3bc7ef0294f5 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:55.297070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:55.311908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:55.313269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:55.313565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:55.313894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] PATH_INFO: `/v3/users/53bf8089e0c1405785ac37ab7fe4ff43` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:55.315652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53bf8089e0c1405785ac37ab7fe4ff43)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:55.318328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:55.374418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53bf8089e0c1405785ac37ab7fe4ff43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.386458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53bf8089e0c1405785ac37ab7fe4ff43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.397064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53bf8089e0c1405785ac37ab7fe4ff43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.404900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53bf8089e0c1405785ac37ab7fe4ff43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.405621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42413ae5-937c-47b4-bf42-efea0d2e7bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53bf8089e0c1405785ac37ab7fe4ff43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.412103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1047/2103] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:36:55 2022] DELETE /identity/v3/users/53bf8089e0c1405785ac37ab7fe4ff43 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:55.708440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:55.710227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:55.727013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993742e-0726-4645-b9a1-a9d31b3a15fb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:55.749451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:55.792730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:55.793609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:55.793894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:55.794177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] PATH_INFO: `/v3/projects/0de36a237cf64be5a5188fe9646a7b7e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:55.795390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de36a237cf64be5a5188fe9646a7b7e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:55.800392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:55.853948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de36a237cf64be5a5188fe9646a7b7e', 'request_id': 'req-9b3bccfe-8f8d-48e4-8269-549391326c1b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.863898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de36a237cf64be5a5188fe9646a7b7e', 'request_id': 'req-9b3bccfe-8f8d-48e4-8269-549391326c1b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.872448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de36a237cf64be5a5188fe9646a7b7e', 'request_id': 'req-9b3bccfe-8f8d-48e4-8269-549391326c1b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.886972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] The token cache is being invalidate because project 0de36a237cf64be5a5188fe9646a7b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:36:55.887220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.888010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9b3bccfe-8f8d-48e4-8269-549391326c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:36:55.889807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1057/2104] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:36:55 2022] DELETE /identity/v3/projects/0de36a237cf64be5a5188fe9646a7b7e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:55.894812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb465a44-b6a5-412a-b33b-0eb184bf19d0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:55.895326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb465a44-b6a5-412a-b33b-0eb184bf19d0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:55.895610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb465a44-b6a5-412a-b33b-0eb184bf19d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:55.914878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-eb465a44-b6a5-412a-b33b-0eb184bf19d0 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:55.916200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eb465a44-b6a5-412a-b33b-0eb184bf19d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:55.991543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1048/2105] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:55.999308 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694295-2768-42d8-a57d-4b5e336c9b2d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:55.999593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694295-2768-42d8-a57d-4b5e336c9b2d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:55.999868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694295-2768-42d8-a57d-4b5e336c9b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.018488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-71694295-2768-42d8-a57d-4b5e336c9b2d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:56.019697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-71694295-2768-42d8-a57d-4b5e336c9b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:56.028841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1058/2106] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:56.033354 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb465a44-b6a5-412a-b33b-0eb184bf19d0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.044340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb465a44-b6a5-412a-b33b-0eb184bf19d0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.049218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245fd8d-5e5d-4d67-a91a-f32820ae8140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.049887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245fd8d-5e5d-4d67-a91a-f32820ae8140 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.050171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245fd8d-5e5d-4d67-a91a-f32820ae8140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.050450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245fd8d-5e5d-4d67-a91a-f32820ae8140 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.051192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245fd8d-5e5d-4d67-a91a-f32820ae8140 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:36:56.051504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245fd8d-5e5d-4d67-a91a-f32820ae8140 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.053976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245fd8d-5e5d-4d67-a91a-f32820ae8140 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.067355 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1049/2107] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:36:56.073249 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694295-2768-42d8-a57d-4b5e336c9b2d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.082693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694295-2768-42d8-a57d-4b5e336c9b2d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.088683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869777e4-cc0b-4b08-86b9-d74e67be952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.089397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869777e4-cc0b-4b08-86b9-d74e67be952d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.089680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869777e4-cc0b-4b08-86b9-d74e67be952d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.089962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869777e4-cc0b-4b08-86b9-d74e67be952d admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.091005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869777e4-cc0b-4b08-86b9-d74e67be952d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.093271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869777e4-cc0b-4b08-86b9-d74e67be952d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.125460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1059/2108] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:56 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:56.129718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245fd8d-5e5d-4d67-a91a-f32820ae8140 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.140050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245fd8d-5e5d-4d67-a91a-f32820ae8140 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.201108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb1709-aedb-4468-a0de-7d627d550129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.201792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb1709-aedb-4468-a0de-7d627d550129 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.202074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb1709-aedb-4468-a0de-7d627d550129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.202352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb1709-aedb-4468-a0de-7d627d550129 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.203052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb1709-aedb-4468-a0de-7d627d550129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.205055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb1709-aedb-4468-a0de-7d627d550129 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.211778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:56.237648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1050/2109] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:56 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:56.238886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869777e4-cc0b-4b08-86b9-d74e67be952d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.249928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869777e4-cc0b-4b08-86b9-d74e67be952d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.256302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db495acb-33d1-4fd6-a08b-a7eb9bef05e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.256990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db495acb-33d1-4fd6-a08b-a7eb9bef05e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.257277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db495acb-33d1-4fd6-a08b-a7eb9bef05e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.257550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db495acb-33d1-4fd6-a08b-a7eb9bef05e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.258291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db495acb-33d1-4fd6-a08b-a7eb9bef05e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.259909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db495acb-33d1-4fd6-a08b-a7eb9bef05e1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.271533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1060/2110] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:36:56.276601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb1709-aedb-4468-a0de-7d627d550129 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.287922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb1709-aedb-4468-a0de-7d627d550129 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.294925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0fb0f-20a6-4863-8bdf-430ce637e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.295605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0fb0f-20a6-4863-8bdf-430ce637e64f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.295889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0fb0f-20a6-4863-8bdf-430ce637e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.296164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0fb0f-20a6-4863-8bdf-430ce637e64f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.296936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0fb0f-20a6-4863-8bdf-430ce637e64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.298551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0fb0f-20a6-4863-8bdf-430ce637e64f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.309480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1051/2111] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:56.313487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db495acb-33d1-4fd6-a08b-a7eb9bef05e1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.324165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db495acb-33d1-4fd6-a08b-a7eb9bef05e1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.329301 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21217dfa-af13-4f16-90e6-4b27c339506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.330222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21217dfa-af13-4f16-90e6-4b27c339506f admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.330515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21217dfa-af13-4f16-90e6-4b27c339506f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.330868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21217dfa-af13-4f16-90e6-4b27c339506f admin admin] PATH_INFO: `/v3/projects/df20dfdf0c4c4e699558521ca1f72cd6/users/f47f2fe9ec944dcab59b090edc6f5e2a/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.350788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21217dfa-af13-4f16-90e6-4b27c339506f admin admin] RBAC: Authorizing `identity:create_grant(project_id=df20dfdf0c4c4e699558521ca1f72cd6, user_id=f47f2fe9ec944dcab59b090edc6f5e2a, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.353534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21217dfa-af13-4f16-90e6-4b27c339506f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.369170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1061/2112] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:56 2022] PUT /identity/v3/projects/df20dfdf0c4c4e699558521ca1f72cd6/users/f47f2fe9ec944dcab59b090edc6f5e2a/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:56.380188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f22685-1185-441b-9371-d6c9d0b41707 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.381443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f22685-1185-441b-9371-d6c9d0b41707 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.382094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f22685-1185-441b-9371-d6c9d0b41707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.410974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-90f22685-1185-441b-9371-d6c9d0b41707 None None] MFA Rules not processed for user `f47f2fe9ec944dcab59b090edc6f5e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:56.412401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-90f22685-1185-441b-9371-d6c9d0b41707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:56.480226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1052/2113] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:56.484932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21217dfa-af13-4f16-90e6-4b27c339506f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.537515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:56.553884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21217dfa-af13-4f16-90e6-4b27c339506f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.558611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443789b-5cc2-4dc9-b15a-38fe44e96852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.559267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443789b-5cc2-4dc9-b15a-38fe44e96852 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.559551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443789b-5cc2-4dc9-b15a-38fe44e96852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.559842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443789b-5cc2-4dc9-b15a-38fe44e96852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.560532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443789b-5cc2-4dc9-b15a-38fe44e96852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.562488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443789b-5cc2-4dc9-b15a-38fe44e96852 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.589633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1062/2114] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:36:56 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:56.594253 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f22685-1185-441b-9371-d6c9d0b41707 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.607273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f22685-1185-441b-9371-d6c9d0b41707 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.669439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5934220-f096-405e-b215-aaef1e82ae11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.670128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5934220-f096-405e-b215-aaef1e82ae11 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.670413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5934220-f096-405e-b215-aaef1e82ae11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.670718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5934220-f096-405e-b215-aaef1e82ae11 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.671435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5934220-f096-405e-b215-aaef1e82ae11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.673442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5934220-f096-405e-b215-aaef1e82ae11 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.696446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1053/2115] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:36:56 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:36:56.700702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443789b-5cc2-4dc9-b15a-38fe44e96852 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.712325 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443789b-5cc2-4dc9-b15a-38fe44e96852 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.717996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511668c-5fe4-4f91-9956-1076a0a37aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.718686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511668c-5fe4-4f91-9956-1076a0a37aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.719008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511668c-5fe4-4f91-9956-1076a0a37aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.719285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511668c-5fe4-4f91-9956-1076a0a37aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.720026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511668c-5fe4-4f91-9956-1076a0a37aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.721759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511668c-5fe4-4f91-9956-1076a0a37aa4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.733288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1063/2116] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:56.737902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5934220-f096-405e-b215-aaef1e82ae11 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.748935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5934220-f096-405e-b215-aaef1e82ae11 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.754215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cd09c-61c1-4dd3-8480-6c22504ab6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.755153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cd09c-61c1-4dd3-8480-6c22504ab6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.755436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cd09c-61c1-4dd3-8480-6c22504ab6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.755741 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cd09c-61c1-4dd3-8480-6c22504ab6f9 admin admin] PATH_INFO: `/v3/projects/41376bcdb5b74dcd8aa65146a6ce3a6d/users/b9af4385d2fe43cd91854c00c984bb9b/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.772472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cd09c-61c1-4dd3-8480-6c22504ab6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41376bcdb5b74dcd8aa65146a6ce3a6d, user_id=b9af4385d2fe43cd91854c00c984bb9b, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.775160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cd09c-61c1-4dd3-8480-6c22504ab6f9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.790576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1054/2117] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:56 2022] PUT /identity/v3/projects/41376bcdb5b74dcd8aa65146a6ce3a6d/users/b9af4385d2fe43cd91854c00c984bb9b/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:56.794836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511668c-5fe4-4f91-9956-1076a0a37aa4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.805513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511668c-5fe4-4f91-9956-1076a0a37aa4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.865487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de01b33-19c9-43ec-91bd-ea29c5727d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.866177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de01b33-19c9-43ec-91bd-ea29c5727d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.866499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de01b33-19c9-43ec-91bd-ea29c5727d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.866775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de01b33-19c9-43ec-91bd-ea29c5727d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.867628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de01b33-19c9-43ec-91bd-ea29c5727d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.869419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de01b33-19c9-43ec-91bd-ea29c5727d60 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.880103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1064/2118] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:36:56.884537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cd09c-61c1-4dd3-8480-6c22504ab6f9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:56.896070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cd09c-61c1-4dd3-8480-6c22504ab6f9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:56.901106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa9ac4-722f-4ce4-9916-ce04715e52eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:56.901993 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa9ac4-722f-4ce4-9916-ce04715e52eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.902278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa9ac4-722f-4ce4-9916-ce04715e52eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.902559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa9ac4-722f-4ce4-9916-ce04715e52eb admin admin] PATH_INFO: `/v3/projects/41376bcdb5b74dcd8aa65146a6ce3a6d/users/b9af4385d2fe43cd91854c00c984bb9b/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.904570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa9ac4-722f-4ce4-9916-ce04715e52eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=41376bcdb5b74dcd8aa65146a6ce3a6d, user_id=b9af4385d2fe43cd91854c00c984bb9b, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:56.907263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa9ac4-722f-4ce4-9916-ce04715e52eb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:56.918152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1055/2119] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:36:56 2022] PUT /identity/v3/projects/41376bcdb5b74dcd8aa65146a6ce3a6d/users/b9af4385d2fe43cd91854c00c984bb9b/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:36:56.928662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77957c6-f6ac-4445-8ec9-5ef52f2acdaf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:56.928984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77957c6-f6ac-4445-8ec9-5ef52f2acdaf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:56.929261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77957c6-f6ac-4445-8ec9-5ef52f2acdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:56.959179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d77957c6-f6ac-4445-8ec9-5ef52f2acdaf None None] MFA Rules not processed for user `b9af4385d2fe43cd91854c00c984bb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:56.960403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d77957c6-f6ac-4445-8ec9-5ef52f2acdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:57.032246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1065/2120] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:56 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:36:57.045409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa9ac4-722f-4ce4-9916-ce04715e52eb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:57.048166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2daa9ac4-722f-4ce4-9916-ce04715e52eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:57.079854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:57.122731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa9ac4-722f-4ce4-9916-ce04715e52eb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:57.127100 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe1857-032b-490d-b8e9-224d84426723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:57.127779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe1857-032b-490d-b8e9-224d84426723 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:57.128059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe1857-032b-490d-b8e9-224d84426723 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:57.128332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe1857-032b-490d-b8e9-224d84426723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:57.139716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe1857-032b-490d-b8e9-224d84426723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:57.141234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe1857-032b-490d-b8e9-224d84426723 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:57.152909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1056/2121] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:57 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:36:57.235377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77957c6-f6ac-4445-8ec9-5ef52f2acdaf None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:36:57.236745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d77957c6-f6ac-4445-8ec9-5ef52f2acdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:57.248997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77957c6-f6ac-4445-8ec9-5ef52f2acdaf None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:36:57.252406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9585ebe-ccf3-4631-8d75-a6b0bc760fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:36:57.253078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9585ebe-ccf3-4631-8d75-a6b0bc760fde service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:57.253361 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9585ebe-ccf3-4631-8d75-a6b0bc760fde service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:57.253636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9585ebe-ccf3-4631-8d75-a6b0bc760fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:57.306213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:36:57.323572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9585ebe-ccf3-4631-8d75-a6b0bc760fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:36:57.325618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9585ebe-ccf3-4631-8d75-a6b0bc760fde service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:36:57.335603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1066/2122] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:36:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:36:59.530715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33e638-4a38-4769-be07-eb8f78c7b929 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:36:59.531470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33e638-4a38-4769-be07-eb8f78c7b929 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:36:59.531470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33e638-4a38-4769-be07-eb8f78c7b929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:36:59.552545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1e33e638-4a38-4769-be07-eb8f78c7b929 None None] MFA Rules not processed for user `57e63c7198d2450698858f9b84393c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:36:59.553813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1e33e638-4a38-4769-be07-eb8f78c7b929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:36:59.610096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1057/2123] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:00.775729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9585ebe-ccf3-4631-8d75-a6b0bc760fde service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:00.777428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c9585ebe-ccf3-4631-8d75-a6b0bc760fde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:00.844803 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9585ebe-ccf3-4631-8d75-a6b0bc760fde service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:00.849679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:00.850574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:00.850891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:00.851172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] PATH_INFO: `/v3/users/e59e20e621424abd99d096c37cccaa21` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:00.852357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59e20e621424abd99d096c37cccaa21)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:00.854478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:00.912565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59e20e621424abd99d096c37cccaa21'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:00.925358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59e20e621424abd99d096c37cccaa21'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:00.935268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59e20e621424abd99d096c37cccaa21'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:00.935974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59e20e621424abd99d096c37cccaa21'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:00.943347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59e20e621424abd99d096c37cccaa21'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:00.949554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1067/2124] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:00 2022] DELETE /identity/v3/users/e59e20e621424abd99d096c37cccaa21 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:00.954178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb99d85-0a32-4c65-9997-262801c1f3ea None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:00.954640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb99d85-0a32-4c65-9997-262801c1f3ea None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:00.954772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb99d85-0a32-4c65-9997-262801c1f3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:00.973407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0fb99d85-0a32-4c65-9997-262801c1f3ea None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:00.974625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0fb99d85-0a32-4c65-9997-262801c1f3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:01.037871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1058/2125] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:01.047703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:01.049330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:01.109089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76614f59-6fb5-4a86-bc33-062bf5ba91c6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:01.113053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f08df-4527-4bcf-8bae-34a3b09c12f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:01.113727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f08df-4527-4bcf-8bae-34a3b09c12f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.114011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f08df-4527-4bcf-8bae-34a3b09c12f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.114287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f08df-4527-4bcf-8bae-34a3b09c12f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.123906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f08df-4527-4bcf-8bae-34a3b09c12f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:01.125380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f08df-4527-4bcf-8bae-34a3b09c12f8 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:01.135179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1068/2126] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:01.449956 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb99d85-0a32-4c65-9997-262801c1f3ea None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:01.451376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0fb99d85-0a32-4c65-9997-262801c1f3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:01.463965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb99d85-0a32-4c65-9997-262801c1f3ea None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:01.468924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:01.469855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.470115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.470386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] PATH_INFO: `/v3/projects/13b14c31a0d34343b8ed80a96bad98f5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.471561 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b14c31a0d34343b8ed80a96bad98f5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:01.473537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:01.527490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b14c31a0d34343b8ed80a96bad98f5', 'request_id': 'req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:01.533767 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b14c31a0d34343b8ed80a96bad98f5', 'request_id': 'req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:01.543380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b14c31a0d34343b8ed80a96bad98f5', 'request_id': 'req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:01.545506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:01.555792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] The token cache is being invalidate because project 13b14c31a0d34343b8ed80a96bad98f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:01.556094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:01.557067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a62979c-b0fb-4520-a73e-b3b0eb90ba87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:01.558669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1059/2127] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:01 2022] DELETE /identity/v3/projects/13b14c31a0d34343b8ed80a96bad98f5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:01.563597 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326640ff-463f-4e17-9392-3bc36fc5e190 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.563883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326640ff-463f-4e17-9392-3bc36fc5e190 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.564194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326640ff-463f-4e17-9392-3bc36fc5e190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.572129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:01.586062 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-326640ff-463f-4e17-9392-3bc36fc5e190 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:01.586062 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-326640ff-463f-4e17-9392-3bc36fc5e190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:01.659475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1069/2128] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:01.659603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460b97f-c802-47b6-947e-8c7306fe0f3f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.659603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460b97f-c802-47b6-947e-8c7306fe0f3f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.659777 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460b97f-c802-47b6-947e-8c7306fe0f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.678307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6460b97f-c802-47b6-947e-8c7306fe0f3f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:01.679664 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6460b97f-c802-47b6-947e-8c7306fe0f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:01.688040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1060/2129] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:37:01.692363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326640ff-463f-4e17-9392-3bc36fc5e190 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:01.702427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326640ff-463f-4e17-9392-3bc36fc5e190 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:01.707697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbee884-0108-40d2-9bcf-4e29c7a3e277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:01.708350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbee884-0108-40d2-9bcf-4e29c7a3e277 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.708633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbee884-0108-40d2-9bcf-4e29c7a3e277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.708945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbee884-0108-40d2-9bcf-4e29c7a3e277 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.709679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbee884-0108-40d2-9bcf-4e29c7a3e277 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:37:01.710131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbee884-0108-40d2-9bcf-4e29c7a3e277 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:01.711744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbee884-0108-40d2-9bcf-4e29c7a3e277 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:01.725231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1070/2130] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:37:01.732337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460b97f-c802-47b6-947e-8c7306fe0f3f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:01.748066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460b97f-c802-47b6-947e-8c7306fe0f3f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:01.754223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b3c14-1b45-4333-9200-3c26a3aba0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:01.754877 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b3c14-1b45-4333-9200-3c26a3aba0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.755151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b3c14-1b45-4333-9200-3c26a3aba0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.755455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b3c14-1b45-4333-9200-3c26a3aba0b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.756142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b3c14-1b45-4333-9200-3c26a3aba0b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:01.758194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b3c14-1b45-4333-9200-3c26a3aba0b2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:01.788083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1061/2131] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:01 2022] POST /identity/v3/projects => generated 359 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:01.792563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbee884-0108-40d2-9bcf-4e29c7a3e277 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:01.803034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbee884-0108-40d2-9bcf-4e29c7a3e277 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:01.859221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789756f6-faa3-40f9-8402-9adb42ee425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:01.859885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789756f6-faa3-40f9-8402-9adb42ee425b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.860166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789756f6-faa3-40f9-8402-9adb42ee425b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.860441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789756f6-faa3-40f9-8402-9adb42ee425b admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.861152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789756f6-faa3-40f9-8402-9adb42ee425b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:01.863070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789756f6-faa3-40f9-8402-9adb42ee425b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:01.884127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1071/2132] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:01 2022] POST /identity/v3/users => generated 280 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:01.889109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b3c14-1b45-4333-9200-3c26a3aba0b2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:01.899954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b3c14-1b45-4333-9200-3c26a3aba0b2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:01.904994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cf7f1-b74d-4cba-b6b3-ba8571e0c3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:01.905651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cf7f1-b74d-4cba-b6b3-ba8571e0c3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.905931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cf7f1-b74d-4cba-b6b3-ba8571e0c3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.906225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cf7f1-b74d-4cba-b6b3-ba8571e0c3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.907046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cf7f1-b74d-4cba-b6b3-ba8571e0c3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:01.908678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cf7f1-b74d-4cba-b6b3-ba8571e0c3e7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:01.919545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1062/2133] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:01.923757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789756f6-faa3-40f9-8402-9adb42ee425b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:01.937511 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789756f6-faa3-40f9-8402-9adb42ee425b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:01.945302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b8abe-07c0-4640-987a-0d947ab45564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:01.945959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b8abe-07c0-4640-987a-0d947ab45564 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.946235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b8abe-07c0-4640-987a-0d947ab45564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.946504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b8abe-07c0-4640-987a-0d947ab45564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.947232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b8abe-07c0-4640-987a-0d947ab45564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:01.948868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b8abe-07c0-4640-987a-0d947ab45564 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:01.959494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1072/2134] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:01.963789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cf7f1-b74d-4cba-b6b3-ba8571e0c3e7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:01.974605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cf7f1-b74d-4cba-b6b3-ba8571e0c3e7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:01.980187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1c0a6-b769-4c7d-a759-c838e4ff2fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:01.981134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1c0a6-b769-4c7d-a759-c838e4ff2fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:01.981427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1c0a6-b769-4c7d-a759-c838e4ff2fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:01.981698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1c0a6-b769-4c7d-a759-c838e4ff2fdb admin admin] PATH_INFO: `/v3/projects/c9ff60bad5a34fb6acfcd686c52a076f/users/e329236afd574826924d48feed991db3/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:01.998566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1c0a6-b769-4c7d-a759-c838e4ff2fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ff60bad5a34fb6acfcd686c52a076f, user_id=e329236afd574826924d48feed991db3, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:02.001322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1c0a6-b769-4c7d-a759-c838e4ff2fdb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:02.017308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1063/2135] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:01 2022] PUT /identity/v3/projects/c9ff60bad5a34fb6acfcd686c52a076f/users/e329236afd574826924d48feed991db3/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:02.028171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3796e6-143f-4ee3-a417-04ef77438fa6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:02.028456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3796e6-143f-4ee3-a417-04ef77438fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:02.028728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3796e6-143f-4ee3-a417-04ef77438fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:02.055033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8a3796e6-143f-4ee3-a417-04ef77438fa6 None None] MFA Rules not processed for user `e329236afd574826924d48feed991db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:02.056236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8a3796e6-143f-4ee3-a417-04ef77438fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:02.115811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:02.119830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1073/2136] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:02 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:02.156113 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1c0a6-b769-4c7d-a759-c838e4ff2fdb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:02.158854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fad1c0a6-b769-4c7d-a759-c838e4ff2fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:02.246078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1c0a6-b769-4c7d-a759-c838e4ff2fdb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:02.250525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9040ce-11a4-43ce-a34f-3ebd13d8a6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:02.251167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9040ce-11a4-43ce-a34f-3ebd13d8a6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:02.251446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9040ce-11a4-43ce-a34f-3ebd13d8a6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:02.251717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9040ce-11a4-43ce-a34f-3ebd13d8a6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:02.261470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9040ce-11a4-43ce-a34f-3ebd13d8a6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:02.262902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9040ce-11a4-43ce-a34f-3ebd13d8a6c2 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:02.272111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1064/2137] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:02 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:02.286546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3796e6-143f-4ee3-a417-04ef77438fa6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:02.352103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3796e6-143f-4ee3-a417-04ef77438fa6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:02.356594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:02.357557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:02.357837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:02.358114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] PATH_INFO: `/v3/users/e329236afd574826924d48feed991db3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:02.359326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e329236afd574826924d48feed991db3)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:02.361307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:02.418405 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e329236afd574826924d48feed991db3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:02.430258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e329236afd574826924d48feed991db3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:02.449859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e329236afd574826924d48feed991db3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:02.450671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e329236afd574826924d48feed991db3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:02.464502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e329236afd574826924d48feed991db3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:02.471455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1074/2138] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:02 2022] DELETE /identity/v3/users/e329236afd574826924d48feed991db3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:02.476454 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e05089-b98d-4b97-bc21-c5dd54d86ec6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:02.476782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e05089-b98d-4b97-bc21-c5dd54d86ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:02.477065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e05089-b98d-4b97-bc21-c5dd54d86ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:02.496341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d8e05089-b98d-4b97-bc21-c5dd54d86ec6 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:02.497608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d8e05089-b98d-4b97-bc21-c5dd54d86ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:02.557347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1065/2139] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:37:02.567294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:02.568776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:02.627213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063db461-b0fd-4205-9c5b-615e8a1822d5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:02.631351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366a4c1-387f-4383-b514-db0e32611fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:02.632041 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366a4c1-387f-4383-b514-db0e32611fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:02.632413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366a4c1-387f-4383-b514-db0e32611fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:02.632723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366a4c1-387f-4383-b514-db0e32611fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:02.642182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366a4c1-387f-4383-b514-db0e32611fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:02.643548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366a4c1-387f-4383-b514-db0e32611fc2 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:02.653256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1075/2140] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:37:02.925538 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e05089-b98d-4b97-bc21-c5dd54d86ec6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:02.927135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d8e05089-b98d-4b97-bc21-c5dd54d86ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:02.940253 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e05089-b98d-4b97-bc21-c5dd54d86ec6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:02.946257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:02.947135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:02.947403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:02.947681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] PATH_INFO: `/v3/projects/c9ff60bad5a34fb6acfcd686c52a076f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:02.948911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ff60bad5a34fb6acfcd686c52a076f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:02.950857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:02.988463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:03.011353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ff60bad5a34fb6acfcd686c52a076f', 'request_id': 'req-996cacde-0e4b-46aa-b2b9-91c5cde00118'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:03.017643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ff60bad5a34fb6acfcd686c52a076f', 'request_id': 'req-996cacde-0e4b-46aa-b2b9-91c5cde00118'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:03.027363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ff60bad5a34fb6acfcd686c52a076f', 'request_id': 'req-996cacde-0e4b-46aa-b2b9-91c5cde00118'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:03.041589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] The token cache is being invalidate because project c9ff60bad5a34fb6acfcd686c52a076f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:03.041860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:03.042555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-996cacde-0e4b-46aa-b2b9-91c5cde00118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:03.044155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1066/2141] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:02 2022] DELETE /identity/v3/projects/c9ff60bad5a34fb6acfcd686c52a076f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:03.049309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ff636-0c4d-4f30-b563-f48383f2357d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.049782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ff636-0c4d-4f30-b563-f48383f2357d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.049873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ff636-0c4d-4f30-b563-f48383f2357d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.068434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5f9ff636-0c4d-4f30-b563-f48383f2357d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:03.069670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5f9ff636-0c4d-4f30-b563-f48383f2357d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:03.088183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:03.134039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1076/2142] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:03.141952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fe91e-0b5f-44c7-ae2d-66eaca590811 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.142237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fe91e-0b5f-44c7-ae2d-66eaca590811 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.142509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fe91e-0b5f-44c7-ae2d-66eaca590811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.162009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e45fe91e-0b5f-44c7-ae2d-66eaca590811 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:03.163266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e45fe91e-0b5f-44c7-ae2d-66eaca590811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:03.171617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1067/2143] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:03.176089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ff636-0c4d-4f30-b563-f48383f2357d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:03.186292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ff636-0c4d-4f30-b563-f48383f2357d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:03.192221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf730eb4-97f4-42e7-9f9d-e351c62ae4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:03.192959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf730eb4-97f4-42e7-9f9d-e351c62ae4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.193253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf730eb4-97f4-42e7-9f9d-e351c62ae4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.193568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf730eb4-97f4-42e7-9f9d-e351c62ae4dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.194462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf730eb4-97f4-42e7-9f9d-e351c62ae4dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:37:03.194778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf730eb4-97f4-42e7-9f9d-e351c62ae4dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:03.196599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf730eb4-97f4-42e7-9f9d-e351c62ae4dc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:03.209203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1077/2144] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:37:03.213868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fe91e-0b5f-44c7-ae2d-66eaca590811 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:03.224646 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fe91e-0b5f-44c7-ae2d-66eaca590811 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:03.229941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a6c3d-8e4a-4b3f-b19f-7dec32618088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:03.230592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a6c3d-8e4a-4b3f-b19f-7dec32618088 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.230874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a6c3d-8e4a-4b3f-b19f-7dec32618088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.231146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a6c3d-8e4a-4b3f-b19f-7dec32618088 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.231834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a6c3d-8e4a-4b3f-b19f-7dec32618088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:03.234149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a6c3d-8e4a-4b3f-b19f-7dec32618088 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:03.266387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1068/2145] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:03 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:03.270859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf730eb4-97f4-42e7-9f9d-e351c62ae4dc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:03.281865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf730eb4-97f4-42e7-9f9d-e351c62ae4dc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:03.348340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc240f9-b52b-4898-854d-7b7362037026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:03.349055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc240f9-b52b-4898-854d-7b7362037026 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.349342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc240f9-b52b-4898-854d-7b7362037026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.349620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc240f9-b52b-4898-854d-7b7362037026 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.350317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc240f9-b52b-4898-854d-7b7362037026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:03.352547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc240f9-b52b-4898-854d-7b7362037026 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:03.374211 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1078/2146] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:03 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:03.378571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a6c3d-8e4a-4b3f-b19f-7dec32618088 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:03.390261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a6c3d-8e4a-4b3f-b19f-7dec32618088 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:03.395824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243de1dd-0205-4f2f-b4f4-bbc878e377dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:03.396583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243de1dd-0205-4f2f-b4f4-bbc878e377dd admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.396976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243de1dd-0205-4f2f-b4f4-bbc878e377dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.397251 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243de1dd-0205-4f2f-b4f4-bbc878e377dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.398177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243de1dd-0205-4f2f-b4f4-bbc878e377dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:03.399708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243de1dd-0205-4f2f-b4f4-bbc878e377dd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:03.411441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1069/2147] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:03.415501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc240f9-b52b-4898-854d-7b7362037026 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:03.425727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc240f9-b52b-4898-854d-7b7362037026 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:03.430468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90001673-01de-45cc-99a9-4eb25e616515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:03.431152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90001673-01de-45cc-99a9-4eb25e616515 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.431474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90001673-01de-45cc-99a9-4eb25e616515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.431774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90001673-01de-45cc-99a9-4eb25e616515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.432464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90001673-01de-45cc-99a9-4eb25e616515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:03.434114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90001673-01de-45cc-99a9-4eb25e616515 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:03.449678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1079/2148] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:03.453992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243de1dd-0205-4f2f-b4f4-bbc878e377dd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:03.467317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243de1dd-0205-4f2f-b4f4-bbc878e377dd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:03.472154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0b388-b0ef-40cb-8ffe-f5821703b257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:03.473118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0b388-b0ef-40cb-8ffe-f5821703b257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.473397 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0b388-b0ef-40cb-8ffe-f5821703b257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.473665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0b388-b0ef-40cb-8ffe-f5821703b257 admin admin] PATH_INFO: `/v3/projects/ac68515ea7c64764b0c9f4005158e360/users/57cee90ceb214b68b07ef3f8ff0b02db/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.494823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0b388-b0ef-40cb-8ffe-f5821703b257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac68515ea7c64764b0c9f4005158e360, user_id=57cee90ceb214b68b07ef3f8ff0b02db, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:03.497549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0b388-b0ef-40cb-8ffe-f5821703b257 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:03.520787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1070/2149] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:03 2022] PUT /identity/v3/projects/ac68515ea7c64764b0c9f4005158e360/users/57cee90ceb214b68b07ef3f8ff0b02db/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:03.527183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02074329-6465-4f30-91b7-48ff4aa176c6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.527183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02074329-6465-4f30-91b7-48ff4aa176c6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.527494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02074329-6465-4f30-91b7-48ff4aa176c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.566699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-02074329-6465-4f30-91b7-48ff4aa176c6 None None] MFA Rules not processed for user `57cee90ceb214b68b07ef3f8ff0b02db`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:03.567929 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-02074329-6465-4f30-91b7-48ff4aa176c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:03.638652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1080/2150] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:03.649866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0b388-b0ef-40cb-8ffe-f5821703b257 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:03.651694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b0e0b388-b0ef-40cb-8ffe-f5821703b257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:03.708667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0b388-b0ef-40cb-8ffe-f5821703b257 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:03.712555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59d9d7-0969-4bd2-b8ad-8c26dfbe0e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:03.713221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59d9d7-0969-4bd2-b8ad-8c26dfbe0e2b service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:03.713502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59d9d7-0969-4bd2-b8ad-8c26dfbe0e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:03.713774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59d9d7-0969-4bd2-b8ad-8c26dfbe0e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:03.723557 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59d9d7-0969-4bd2-b8ad-8c26dfbe0e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:03.725036 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59d9d7-0969-4bd2-b8ad-8c26dfbe0e2b service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:03.735413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1071/2151] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:05.130540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdba7f-eb79-49a3-b695-6c5f7071a961 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:05.131093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdba7f-eb79-49a3-b695-6c5f7071a961 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:05.131093 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdba7f-eb79-49a3-b695-6c5f7071a961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:05.150388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c8fdba7f-eb79-49a3-b695-6c5f7071a961 None None] MFA Rules not processed for user `57cee90ceb214b68b07ef3f8ff0b02db`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:05.151830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c8fdba7f-eb79-49a3-b695-6c5f7071a961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:05.173790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1081/2152] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:07.129384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59d9d7-0969-4bd2-b8ad-8c26dfbe0e2b service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:07.131056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eb59d9d7-0969-4bd2-b8ad-8c26dfbe0e2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:07.186147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:07.201665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59d9d7-0969-4bd2-b8ad-8c26dfbe0e2b service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:07.206984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:07.207854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:07.208130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:07.208395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] PATH_INFO: `/v3/users/96f5680d4d4c4c34b8e8759efaf64515` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:07.209622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f5680d4d4c4c34b8e8759efaf64515)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:07.211699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:07.267911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f5680d4d4c4c34b8e8759efaf64515'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.282919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f5680d4d4c4c34b8e8759efaf64515'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.292076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f5680d4d4c4c34b8e8759efaf64515'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.299734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f5680d4d4c4c34b8e8759efaf64515'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.300453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f5680d4d4c4c34b8e8759efaf64515'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.307240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1072/2153] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:07 2022] DELETE /identity/v3/users/96f5680d4d4c4c34b8e8759efaf64515 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:07.312068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb985db-bbaa-44b7-a7e7-acf50905bd62 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:07.312554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb985db-bbaa-44b7-a7e7-acf50905bd62 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:07.312647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb985db-bbaa-44b7-a7e7-acf50905bd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:07.332250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0fb985db-bbaa-44b7-a7e7-acf50905bd62 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:07.333486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0fb985db-bbaa-44b7-a7e7-acf50905bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:07.339733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:07.413080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1082/2154] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:07.422592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:07.423989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:07.496245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d956d-8fba-4974-9ae8-99aeafdfd66b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:07.500411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7bfbf-d51f-40f0-9a37-8a8c899f19fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:07.501180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7bfbf-d51f-40f0-9a37-8a8c899f19fb service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:07.501529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7bfbf-d51f-40f0-9a37-8a8c899f19fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:07.501847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7bfbf-d51f-40f0-9a37-8a8c899f19fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:07.512362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7bfbf-d51f-40f0-9a37-8a8c899f19fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:07.513943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7bfbf-d51f-40f0-9a37-8a8c899f19fb service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:07.524480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1073/2155] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:07.773665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb985db-bbaa-44b7-a7e7-acf50905bd62 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:07.775725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0fb985db-bbaa-44b7-a7e7-acf50905bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:07.790418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb985db-bbaa-44b7-a7e7-acf50905bd62 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:07.797415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:07.798586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:07.799165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:07.800299 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] PATH_INFO: `/v3/users/f47f2fe9ec944dcab59b090edc6f5e2a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:07.802296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f47f2fe9ec944dcab59b090edc6f5e2a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:07.805993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:07.887160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f47f2fe9ec944dcab59b090edc6f5e2a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.915760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f47f2fe9ec944dcab59b090edc6f5e2a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.933884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f47f2fe9ec944dcab59b090edc6f5e2a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.934616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f47f2fe9ec944dcab59b090edc6f5e2a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.944655 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f47f2fe9ec944dcab59b090edc6f5e2a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:07.965669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1083/2156] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:07 2022] DELETE /identity/v3/users/f47f2fe9ec944dcab59b090edc6f5e2a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:07.969551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66567051-ad97-4854-8e8a-26e7e4e4554d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:07.969551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66567051-ad97-4854-8e8a-26e7e4e4554d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:07.969551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66567051-ad97-4854-8e8a-26e7e4e4554d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:07.987756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-66567051-ad97-4854-8e8a-26e7e4e4554d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:07.989384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-66567051-ad97-4854-8e8a-26e7e4e4554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:08.032104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:08.033771 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:08.101907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1074/2157] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:08.121432 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66567051-ad97-4854-8e8a-26e7e4e4554d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:08.122988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-66567051-ad97-4854-8e8a-26e7e4e4554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:08.157550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:08.168665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c536e-f82d-4f4a-8483-765f7158a424 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:08.182077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:08.182510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.182827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.183071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] PATH_INFO: `/v3/projects/68207b0d0b3c48ceabc7828d0db8825f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.189280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68207b0d0b3c48ceabc7828d0db8825f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:08.194623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:08.210846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66567051-ad97-4854-8e8a-26e7e4e4554d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:08.219027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1589cba-ae4c-4db1-8561-bbf191d1682c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:08.219685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1589cba-ae4c-4db1-8561-bbf191d1682c service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.219967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1589cba-ae4c-4db1-8561-bbf191d1682c service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.221166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1589cba-ae4c-4db1-8561-bbf191d1682c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.234323 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1589cba-ae4c-4db1-8561-bbf191d1682c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:08.236687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1589cba-ae4c-4db1-8561-bbf191d1682c service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:08.248752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:08.252405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1075/2158] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:08.271964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68207b0d0b3c48ceabc7828d0db8825f', 'request_id': 'req-e683d6ba-38e1-4097-a49b-1187b45c89f7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:08.282095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68207b0d0b3c48ceabc7828d0db8825f', 'request_id': 'req-e683d6ba-38e1-4097-a49b-1187b45c89f7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:08.290358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68207b0d0b3c48ceabc7828d0db8825f', 'request_id': 'req-e683d6ba-38e1-4097-a49b-1187b45c89f7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:08.304671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] The token cache is being invalidate because project 68207b0d0b3c48ceabc7828d0db8825f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:08.305307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:08.306150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e683d6ba-38e1-4097-a49b-1187b45c89f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:08.308227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1084/2159] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:08 2022] DELETE /identity/v3/projects/68207b0d0b3c48ceabc7828d0db8825f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:08.314109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3871c-7fad-4cc6-9f3a-506af1308a44 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.314467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3871c-7fad-4cc6-9f3a-506af1308a44 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.314815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3871c-7fad-4cc6-9f3a-506af1308a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.335236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7db3871c-7fad-4cc6-9f3a-506af1308a44 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:08.336657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7db3871c-7fad-4cc6-9f3a-506af1308a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:08.414048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1076/2160] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:08.423165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20593af5-650c-4ed8-b2fb-8b547d242bfd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.423723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20593af5-650c-4ed8-b2fb-8b547d242bfd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.423900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20593af5-650c-4ed8-b2fb-8b547d242bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.448200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-20593af5-650c-4ed8-b2fb-8b547d242bfd None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:08.450087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-20593af5-650c-4ed8-b2fb-8b547d242bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:08.461139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1085/2161] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:08.464995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3871c-7fad-4cc6-9f3a-506af1308a44 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:08.481675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3871c-7fad-4cc6-9f3a-506af1308a44 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:08.489038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a49352-1019-4982-bbcb-94d355a47194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:08.489912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a49352-1019-4982-bbcb-94d355a47194 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.490168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a49352-1019-4982-bbcb-94d355a47194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.490489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a49352-1019-4982-bbcb-94d355a47194 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.491807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a49352-1019-4982-bbcb-94d355a47194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:37:08.492154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a49352-1019-4982-bbcb-94d355a47194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:08.494210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a49352-1019-4982-bbcb-94d355a47194 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:08.515825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1077/2162] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:37:08.519395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20593af5-650c-4ed8-b2fb-8b547d242bfd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:08.534677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20593af5-650c-4ed8-b2fb-8b547d242bfd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:08.543788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddd0d4-f3c0-4116-80f8-33b10e8a4404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:08.544536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddd0d4-f3c0-4116-80f8-33b10e8a4404 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.545379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddd0d4-f3c0-4116-80f8-33b10e8a4404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.545927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddd0d4-f3c0-4116-80f8-33b10e8a4404 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.546837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddd0d4-f3c0-4116-80f8-33b10e8a4404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:08.549865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddd0d4-f3c0-4116-80f8-33b10e8a4404 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:08.588783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1086/2163] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:08 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:08.591310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a49352-1019-4982-bbcb-94d355a47194 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:08.620211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a49352-1019-4982-bbcb-94d355a47194 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:08.719157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41853b-6c24-41f7-8212-0a6549a4910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:08.719871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41853b-6c24-41f7-8212-0a6549a4910c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.720207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41853b-6c24-41f7-8212-0a6549a4910c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.720566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41853b-6c24-41f7-8212-0a6549a4910c admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.721436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41853b-6c24-41f7-8212-0a6549a4910c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:08.723471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41853b-6c24-41f7-8212-0a6549a4910c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:08.749700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1078/2164] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:08 2022] POST /identity/v3/users => generated 288 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:08.753718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddd0d4-f3c0-4116-80f8-33b10e8a4404 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:08.765409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddd0d4-f3c0-4116-80f8-33b10e8a4404 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:08.770812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40683ada-c4ae-4ca6-b4d6-3045192a7079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:08.771677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40683ada-c4ae-4ca6-b4d6-3045192a7079 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.772065 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40683ada-c4ae-4ca6-b4d6-3045192a7079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.772464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40683ada-c4ae-4ca6-b4d6-3045192a7079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.773397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40683ada-c4ae-4ca6-b4d6-3045192a7079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:08.775993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40683ada-c4ae-4ca6-b4d6-3045192a7079 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:08.788953 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1087/2165] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:37:08.793673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41853b-6c24-41f7-8212-0a6549a4910c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:08.804579 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41853b-6c24-41f7-8212-0a6549a4910c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:08.810408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433de4b-ba57-4ba0-8dd4-efe88568d5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:08.811122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433de4b-ba57-4ba0-8dd4-efe88568d5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.811412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433de4b-ba57-4ba0-8dd4-efe88568d5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.811698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433de4b-ba57-4ba0-8dd4-efe88568d5a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.812477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433de4b-ba57-4ba0-8dd4-efe88568d5a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:08.814198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433de4b-ba57-4ba0-8dd4-efe88568d5a5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:08.827186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1079/2166] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:08.831524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40683ada-c4ae-4ca6-b4d6-3045192a7079 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:08.845742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40683ada-c4ae-4ca6-b4d6-3045192a7079 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:08.851021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe91e6-ef9b-4da3-8917-5a490567c1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:08.852084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe91e6-ef9b-4da3-8917-5a490567c1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.852612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe91e6-ef9b-4da3-8917-5a490567c1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.852612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe91e6-ef9b-4da3-8917-5a490567c1f3 admin admin] PATH_INFO: `/v3/projects/7c8f7705b24b412e903df811fee2320e/users/fcd289bcf55340f6b80b62bf3dc7d513/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.872393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe91e6-ef9b-4da3-8917-5a490567c1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c8f7705b24b412e903df811fee2320e, user_id=fcd289bcf55340f6b80b62bf3dc7d513, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:08.875381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe91e6-ef9b-4da3-8917-5a490567c1f3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:08.897972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1088/2167] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:08 2022] PUT /identity/v3/projects/7c8f7705b24b412e903df811fee2320e/users/fcd289bcf55340f6b80b62bf3dc7d513/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:08.905802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189e81b-a5e4-4178-88d6-8ae6dabadfd6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:08.906140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189e81b-a5e4-4178-88d6-8ae6dabadfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:08.906370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189e81b-a5e4-4178-88d6-8ae6dabadfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:08.942036 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7189e81b-a5e4-4178-88d6-8ae6dabadfd6 None None] MFA Rules not processed for user `fcd289bcf55340f6b80b62bf3dc7d513`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:08.943555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7189e81b-a5e4-4178-88d6-8ae6dabadfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:08.959306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:09.031936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1080/2168] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:09.037063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe91e6-ef9b-4da3-8917-5a490567c1f3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:09.069811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189e81b-a5e4-4178-88d6-8ae6dabadfd6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:09.073582 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7189e81b-a5e4-4178-88d6-8ae6dabadfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:09.117734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe91e6-ef9b-4da3-8917-5a490567c1f3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:09.125633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf829ba2-cf1d-447f-ad74-b12d40f406d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:09.126705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf829ba2-cf1d-447f-ad74-b12d40f406d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:09.127215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf829ba2-cf1d-447f-ad74-b12d40f406d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:09.127653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf829ba2-cf1d-447f-ad74-b12d40f406d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:09.128649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf829ba2-cf1d-447f-ad74-b12d40f406d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:09.134796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf829ba2-cf1d-447f-ad74-b12d40f406d6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:09.179866 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1089/2169] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:09 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:09.181125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf829ba2-cf1d-447f-ad74-b12d40f406d6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:09.198697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189e81b-a5e4-4178-88d6-8ae6dabadfd6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:09.225941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:09.268526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf829ba2-cf1d-447f-ad74-b12d40f406d6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:09.274607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba918c-f0d5-4371-9a43-31882394da2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:09.275287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba918c-f0d5-4371-9a43-31882394da2a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:09.275834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba918c-f0d5-4371-9a43-31882394da2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:09.276173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba918c-f0d5-4371-9a43-31882394da2a admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:09.276871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba918c-f0d5-4371-9a43-31882394da2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:09.279160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba918c-f0d5-4371-9a43-31882394da2a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:09.292129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:09.293295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:09.293746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:09.294162 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] PATH_INFO: `/v3/projects/df20dfdf0c4c4e699558521ca1f72cd6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:09.296543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df20dfdf0c4c4e699558521ca1f72cd6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:09.299606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:09.317349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1090/2170] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:09 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:09.320270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba918c-f0d5-4371-9a43-31882394da2a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:09.333736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba918c-f0d5-4371-9a43-31882394da2a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:09.340247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c49726-5cbe-4a5f-9bf4-375cd2ff9161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:09.341474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c49726-5cbe-4a5f-9bf4-375cd2ff9161 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:09.341879 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c49726-5cbe-4a5f-9bf4-375cd2ff9161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:09.342259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c49726-5cbe-4a5f-9bf4-375cd2ff9161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:09.343208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c49726-5cbe-4a5f-9bf4-375cd2ff9161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:09.344971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c49726-5cbe-4a5f-9bf4-375cd2ff9161 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:09.361371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1091/2171] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:37:09.365278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c49726-5cbe-4a5f-9bf4-375cd2ff9161 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:09.371548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df20dfdf0c4c4e699558521ca1f72cd6', 'request_id': 'req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.377027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c49726-5cbe-4a5f-9bf4-375cd2ff9161 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:09.381884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df20dfdf0c4c4e699558521ca1f72cd6', 'request_id': 'req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.394834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df20dfdf0c4c4e699558521ca1f72cd6', 'request_id': 'req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.397504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:09.408560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] The token cache is being invalidate because project df20dfdf0c4c4e699558521ca1f72cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:09.409549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.411643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.416865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1081/2172] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:09 2022] DELETE /identity/v3/projects/df20dfdf0c4c4e699558521ca1f72cd6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:09.417801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:09.419202 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:09.447165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5b6ca-eee6-45b1-84d0-74f964b7c18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:09.447872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5b6ca-eee6-45b1-84d0-74f964b7c18e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:09.448139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5b6ca-eee6-45b1-84d0-74f964b7c18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:09.448389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5b6ca-eee6-45b1-84d0-74f964b7c18e admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:09.449172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5b6ca-eee6-45b1-84d0-74f964b7c18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:09.450826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5b6ca-eee6-45b1-84d0-74f964b7c18e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:09.466173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1092/2173] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:09.470430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5b6ca-eee6-45b1-84d0-74f964b7c18e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:09.472230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bdd5b6ca-eee6-45b1-84d0-74f964b7c18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:09.485642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5b6ca-eee6-45b1-84d0-74f964b7c18e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:09.491173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c5e87-fb11-4a98-9561-69d7bd12da3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:09.492079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c5e87-fb11-4a98-9561-69d7bd12da3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:09.492364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c5e87-fb11-4a98-9561-69d7bd12da3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:09.492646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c5e87-fb11-4a98-9561-69d7bd12da3c admin admin] PATH_INFO: `/v3/projects/5bebca6bf6c541cfa60bb506c26d0b44/users/8af0c8c971ea4b339c0b74dd071131f8/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:09.495777 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53af10b-2fd1-40f4-a8b3-db8ec6d09173 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:09.501678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:09.502591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:09.502871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:09.503125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] PATH_INFO: `/v3/users/b9af4385d2fe43cd91854c00c984bb9b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:09.504376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9af4385d2fe43cd91854c00c984bb9b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:09.506359 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:09.509325 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c5e87-fb11-4a98-9561-69d7bd12da3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bebca6bf6c541cfa60bb506c26d0b44, user_id=8af0c8c971ea4b339c0b74dd071131f8, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:09.512104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c5e87-fb11-4a98-9561-69d7bd12da3c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:09.529605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1093/2174] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:09 2022] PUT /identity/v3/projects/5bebca6bf6c541cfa60bb506c26d0b44/users/8af0c8c971ea4b339c0b74dd071131f8/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:09.540496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d16805-d4bb-4b76-9154-4ded1fd33440 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:09.541348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d16805-d4bb-4b76-9154-4ded1fd33440 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:09.541619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d16805-d4bb-4b76-9154-4ded1fd33440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:09.563019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9af4385d2fe43cd91854c00c984bb9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.568718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f9d16805-d4bb-4b76-9154-4ded1fd33440 None None] MFA Rules not processed for user `8af0c8c971ea4b339c0b74dd071131f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:09.569998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f9d16805-d4bb-4b76-9154-4ded1fd33440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:09.575735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9af4385d2fe43cd91854c00c984bb9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.585643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9af4385d2fe43cd91854c00c984bb9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.603737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9af4385d2fe43cd91854c00c984bb9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.604586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9af4385d2fe43cd91854c00c984bb9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:09.611628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1082/2175] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:09 2022] DELETE /identity/v3/users/b9af4385d2fe43cd91854c00c984bb9b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:09.666464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1094/2176] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:37:09.672342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:09.674273 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:09.785127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98647e-3ad0-4c21-ae9c-61a2242ada4b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:09.792119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce992d-b362-4c69-bc82-02b5ab402ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:09.793834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce992d-b362-4c69-bc82-02b5ab402ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:09.794237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce992d-b362-4c69-bc82-02b5ab402ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:09.794604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce992d-b362-4c69-bc82-02b5ab402ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:09.797255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce992d-b362-4c69-bc82-02b5ab402ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:09.801233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce992d-b362-4c69-bc82-02b5ab402ee6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:09.841613 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:09.865419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1083/2177] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:09 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:09.870089 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d16805-d4bb-4b76-9154-4ded1fd33440 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:09.896451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:09.962013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d16805-d4bb-4b76-9154-4ded1fd33440 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.072295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e75c5-ae4a-4691-ab93-8b5140580f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.073214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e75c5-ae4a-4691-ab93-8b5140580f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.073422 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e75c5-ae4a-4691-ab93-8b5140580f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.073739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e75c5-ae4a-4691-ab93-8b5140580f81 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.074615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e75c5-ae4a-4691-ab93-8b5140580f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.076767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e75c5-ae4a-4691-ab93-8b5140580f81 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.092911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce992d-b362-4c69-bc82-02b5ab402ee6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.095502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-52ce992d-b362-4c69-bc82-02b5ab402ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:10.116130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce992d-b362-4c69-bc82-02b5ab402ee6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.123499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.124555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.124924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.125276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] PATH_INFO: `/v3/projects/41376bcdb5b74dcd8aa65146a6ce3a6d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.126947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41376bcdb5b74dcd8aa65146a6ce3a6d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.129373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.141445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1095/2178] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:09 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:37:10.143007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e75c5-ae4a-4691-ab93-8b5140580f81 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.181744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e75c5-ae4a-4691-ab93-8b5140580f81 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.189096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d6ffe-a819-4670-89fd-361f6f2564f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.192949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d6ffe-a819-4670-89fd-361f6f2564f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.193258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d6ffe-a819-4670-89fd-361f6f2564f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.193536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d6ffe-a819-4670-89fd-361f6f2564f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.196472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d6ffe-a819-4670-89fd-361f6f2564f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.198433 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d6ffe-a819-4670-89fd-361f6f2564f7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.202570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41376bcdb5b74dcd8aa65146a6ce3a6d', 'request_id': 'req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:10.210268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41376bcdb5b74dcd8aa65146a6ce3a6d', 'request_id': 'req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:10.224447 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41376bcdb5b74dcd8aa65146a6ce3a6d', 'request_id': 'req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:10.239691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1096/2179] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:10.244520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] The token cache is being invalidate because project 41376bcdb5b74dcd8aa65146a6ce3a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:10.244725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d6ffe-a819-4670-89fd-361f6f2564f7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.244846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:10.245982 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e7d4d20d-2b9a-4897-95cc-a6c865e033d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:10.250159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1084/2180] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:10 2022] DELETE /identity/v3/projects/41376bcdb5b74dcd8aa65146a6ce3a6d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:10.254885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886a4a6-8142-4064-8e17-c7f9f2d3fb37 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.255462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886a4a6-8142-4064-8e17-c7f9f2d3fb37 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.255562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886a4a6-8142-4064-8e17-c7f9f2d3fb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.266678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d6ffe-a819-4670-89fd-361f6f2564f7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.294023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f886a4a6-8142-4064-8e17-c7f9f2d3fb37 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:10.295937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f886a4a6-8142-4064-8e17-c7f9f2d3fb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:10.419656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ffaa7-2b37-41c8-9e43-ed6199484f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.420577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ffaa7-2b37-41c8-9e43-ed6199484f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.420907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ffaa7-2b37-41c8-9e43-ed6199484f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.421194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ffaa7-2b37-41c8-9e43-ed6199484f9f admin admin] PATH_INFO: `/v3/projects/bc67e022e8ae4cceb3da72e12bf8899e/users/fc24abe470894bf78b11dfe31b1d5296/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.421621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1085/2181] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:10.449924 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7a74f-4a52-438e-ab70-253ca55c4bb2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.451334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7a74f-4a52-438e-ab70-253ca55c4bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.451830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7a74f-4a52-438e-ab70-253ca55c4bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.458077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ffaa7-2b37-41c8-9e43-ed6199484f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc67e022e8ae4cceb3da72e12bf8899e, user_id=fc24abe470894bf78b11dfe31b1d5296, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.461018 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ffaa7-2b37-41c8-9e43-ed6199484f9f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.483725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-36f7a74f-4a52-438e-ab70-253ca55c4bb2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:10.485453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1097/2182] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:10 2022] PUT /identity/v3/projects/bc67e022e8ae4cceb3da72e12bf8899e/users/fc24abe470894bf78b11dfe31b1d5296/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:10.489371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ffaa7-2b37-41c8-9e43-ed6199484f9f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.489704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-36f7a74f-4a52-438e-ab70-253ca55c4bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:10.491348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4a5ffaa7-2b37-41c8-9e43-ed6199484f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:10.533080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:10.577225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1086/2183] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:10.578529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7a74f-4a52-438e-ab70-253ca55c4bb2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.598553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7a74f-4a52-438e-ab70-253ca55c4bb2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.610651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5819975-6fce-43cf-b4a3-b7baffed84a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.611336 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5819975-6fce-43cf-b4a3-b7baffed84a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.611614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5819975-6fce-43cf-b4a3-b7baffed84a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.611886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5819975-6fce-43cf-b4a3-b7baffed84a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.612846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5819975-6fce-43cf-b4a3-b7baffed84a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:37:10.613482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5819975-6fce-43cf-b4a3-b7baffed84a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.615263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5819975-6fce-43cf-b4a3-b7baffed84a3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.617472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ffaa7-2b37-41c8-9e43-ed6199484f9f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.624451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40dcbd-4577-430f-aa33-8fa79fc12956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.625313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40dcbd-4577-430f-aa33-8fa79fc12956 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.625659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40dcbd-4577-430f-aa33-8fa79fc12956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.626031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40dcbd-4577-430f-aa33-8fa79fc12956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.627210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40dcbd-4577-430f-aa33-8fa79fc12956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.629353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40dcbd-4577-430f-aa33-8fa79fc12956 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.645535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1098/2184] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:37:10.648617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40dcbd-4577-430f-aa33-8fa79fc12956 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.650967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1087/2185] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:37:10.652652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5819975-6fce-43cf-b4a3-b7baffed84a3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.662019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40dcbd-4577-430f-aa33-8fa79fc12956 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.668630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec0b48-7750-494e-8444-bb8a55e1e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.669648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec0b48-7750-494e-8444-bb8a55e1e858 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.670039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec0b48-7750-494e-8444-bb8a55e1e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.670576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec0b48-7750-494e-8444-bb8a55e1e858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.671541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec0b48-7750-494e-8444-bb8a55e1e858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.673818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec0b48-7750-494e-8444-bb8a55e1e858 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.676410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5819975-6fce-43cf-b4a3-b7baffed84a3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.690372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9412b-a117-4f81-805d-db7664146cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.691291 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9412b-a117-4f81-805d-db7664146cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.691570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9412b-a117-4f81-805d-db7664146cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.691845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9412b-a117-4f81-805d-db7664146cb8 admin admin] PATH_INFO: `/v3/projects/bc67e022e8ae4cceb3da72e12bf8899e/users/fc24abe470894bf78b11dfe31b1d5296/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.694443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9412b-a117-4f81-805d-db7664146cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc67e022e8ae4cceb3da72e12bf8899e, user_id=fc24abe470894bf78b11dfe31b1d5296, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.697559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9412b-a117-4f81-805d-db7664146cb8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.700133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:10.723628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1099/2186] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:10 2022] POST /identity/v3/projects => generated 369 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:10.727739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec0b48-7750-494e-8444-bb8a55e1e858 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.733040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1088/2187] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:10 2022] PUT /identity/v3/projects/bc67e022e8ae4cceb3da72e12bf8899e/users/fc24abe470894bf78b11dfe31b1d5296/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:10.742048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02df0fe-0286-4901-9fc0-61f10cb87b2a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.742342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02df0fe-0286-4901-9fc0-61f10cb87b2a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.742611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02df0fe-0286-4901-9fc0-61f10cb87b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.787498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a02df0fe-0286-4901-9fc0-61f10cb87b2a None None] MFA Rules not processed for user `fc24abe470894bf78b11dfe31b1d5296`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:10.789757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a02df0fe-0286-4901-9fc0-61f10cb87b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:10.828919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec0b48-7750-494e-8444-bb8a55e1e858 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.834328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e33d50-d5a4-4042-bccf-59940172024e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.835012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e33d50-d5a4-4042-bccf-59940172024e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.835300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e33d50-d5a4-4042-bccf-59940172024e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.835579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e33d50-d5a4-4042-bccf-59940172024e admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.836835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e33d50-d5a4-4042-bccf-59940172024e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.839530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e33d50-d5a4-4042-bccf-59940172024e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.868724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1100/2188] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:10 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:37:10.869955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e33d50-d5a4-4042-bccf-59940172024e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.878338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1089/2189] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:37:10.879763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e33d50-d5a4-4042-bccf-59940172024e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.884565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27d3f1-2037-47a0-8a2e-1279e3170402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.885245 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27d3f1-2037-47a0-8a2e-1279e3170402 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.885551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27d3f1-2037-47a0-8a2e-1279e3170402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.885822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27d3f1-2037-47a0-8a2e-1279e3170402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.886698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27d3f1-2037-47a0-8a2e-1279e3170402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.888328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27d3f1-2037-47a0-8a2e-1279e3170402 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.890060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02df0fe-0286-4901-9fc0-61f10cb87b2a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.891411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a02df0fe-0286-4901-9fc0-61f10cb87b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:10.899755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1101/2190] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:37:10.903502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27d3f1-2037-47a0-8a2e-1279e3170402 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.913158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27d3f1-2037-47a0-8a2e-1279e3170402 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.918259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35845a6e-873c-484d-a3ac-a0e0289f4b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.918918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35845a6e-873c-484d-a3ac-a0e0289f4b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.919382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35845a6e-873c-484d-a3ac-a0e0289f4b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.919517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35845a6e-873c-484d-a3ac-a0e0289f4b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.920291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35845a6e-873c-484d-a3ac-a0e0289f4b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.921930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35845a6e-873c-484d-a3ac-a0e0289f4b3b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.932674 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1102/2191] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:37:10.936686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35845a6e-873c-484d-a3ac-a0e0289f4b3b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:10.946518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35845a6e-873c-484d-a3ac-a0e0289f4b3b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.948236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02df0fe-0286-4901-9fc0-61f10cb87b2a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:10.951229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05d2a7-3e1a-41b7-bf00-cf2e4cf78b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.952028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a49969-f494-4e6e-a33f-1362fc1bafe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:10.952149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05d2a7-3e1a-41b7-bf00-cf2e4cf78b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.952364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05d2a7-3e1a-41b7-bf00-cf2e4cf78b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.952637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05d2a7-3e1a-41b7-bf00-cf2e4cf78b0c admin admin] PATH_INFO: `/v3/projects/f36a54ecd3744e18ab8bca58bfe9e0d0/users/acc80d06d3684b588b4322a34dd70f86/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.952771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a49969-f494-4e6e-a33f-1362fc1bafe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.953059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a49969-f494-4e6e-a33f-1362fc1bafe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.953329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a49969-f494-4e6e-a33f-1362fc1bafe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:10.955150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-53a49969-f494-4e6e-a33f-1362fc1bafe3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:10.968192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05d2a7-3e1a-41b7-bf00-cf2e4cf78b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36a54ecd3744e18ab8bca58bfe9e0d0, user_id=acc80d06d3684b588b4322a34dd70f86, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:10.970869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05d2a7-3e1a-41b7-bf00-cf2e4cf78b0c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:10.986869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1103/2192] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:10 2022] PUT /identity/v3/projects/f36a54ecd3744e18ab8bca58bfe9e0d0/users/acc80d06d3684b588b4322a34dd70f86/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:10.998053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d005e5-ce5e-48c1-9386-de5124e5291f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:10.998338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d005e5-ce5e-48c1-9386-de5124e5291f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:10.998627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d005e5-ce5e-48c1-9386-de5124e5291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.016355 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a49969-f494-4e6e-a33f-1362fc1bafe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:11.017860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a49969-f494-4e6e-a33f-1362fc1bafe3 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:11.027347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1090/2193] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:37:11.030363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-19d005e5-ce5e-48c1-9386-de5124e5291f None None] MFA Rules not processed for user `acc80d06d3684b588b4322a34dd70f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:11.032902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-19d005e5-ce5e-48c1-9386-de5124e5291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:11.065544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:11.105634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1104/2194] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:37:11.106290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a49969-f494-4e6e-a33f-1362fc1bafe3 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:11.125495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a49969-f494-4e6e-a33f-1362fc1bafe3 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:11.171007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:11.200367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e2705-d8e8-437c-8338-4c8559960c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:11.201142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e2705-d8e8-437c-8338-4c8559960c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:11.201430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e2705-d8e8-437c-8338-4c8559960c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:11.201699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e2705-d8e8-437c-8338-4c8559960c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.202394 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e2705-d8e8-437c-8338-4c8559960c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:11.204324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e2705-d8e8-437c-8338-4c8559960c73 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:11.236510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1091/2195] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:11 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:37:11.237503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d005e5-ce5e-48c1-9386-de5124e5291f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:11.248627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d005e5-ce5e-48c1-9386-de5124e5291f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:11.316120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd0c10-4519-4597-bf66-7812b6bc7ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:11.316846 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd0c10-4519-4597-bf66-7812b6bc7ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:11.317134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd0c10-4519-4597-bf66-7812b6bc7ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:11.317412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd0c10-4519-4597-bf66-7812b6bc7ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.318136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd0c10-4519-4597-bf66-7812b6bc7ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:11.320107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd0c10-4519-4597-bf66-7812b6bc7ed7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:11.343147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1105/2196] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:11 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:11.347001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e2705-d8e8-437c-8338-4c8559960c73 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:11.358230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e2705-d8e8-437c-8338-4c8559960c73 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:11.363812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d313c1e-69a8-46f4-8bbb-a5c3f11f4576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:11.364665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d313c1e-69a8-46f4-8bbb-a5c3f11f4576 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:11.365054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d313c1e-69a8-46f4-8bbb-a5c3f11f4576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:11.365383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d313c1e-69a8-46f4-8bbb-a5c3f11f4576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.366228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d313c1e-69a8-46f4-8bbb-a5c3f11f4576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:11.368272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d313c1e-69a8-46f4-8bbb-a5c3f11f4576 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:11.380639 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1092/2197] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:11.385442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd0c10-4519-4597-bf66-7812b6bc7ed7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:11.395989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd0c10-4519-4597-bf66-7812b6bc7ed7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:11.401010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571b2ee-7daa-40bd-aff7-fa3741bdcfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:11.401921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571b2ee-7daa-40bd-aff7-fa3741bdcfb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:11.402200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571b2ee-7daa-40bd-aff7-fa3741bdcfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:11.402479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571b2ee-7daa-40bd-aff7-fa3741bdcfb9 admin admin] PATH_INFO: `/v3/projects/efe81245e8654a5d80332e40f9842ee0/users/f1a9583b8a824b0ba7148cca920c7c3c/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.418552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571b2ee-7daa-40bd-aff7-fa3741bdcfb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe81245e8654a5d80332e40f9842ee0, user_id=f1a9583b8a824b0ba7148cca920c7c3c, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:11.421421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571b2ee-7daa-40bd-aff7-fa3741bdcfb9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:11.435789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1106/2198] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:11 2022] PUT /identity/v3/projects/efe81245e8654a5d80332e40f9842ee0/users/f1a9583b8a824b0ba7148cca920c7c3c/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:11.440024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d313c1e-69a8-46f4-8bbb-a5c3f11f4576 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:11.450606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d313c1e-69a8-46f4-8bbb-a5c3f11f4576 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:11.512921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b2329-5d33-4510-8527-7b7a56f89f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:11.513599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b2329-5d33-4510-8527-7b7a56f89f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:11.513882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b2329-5d33-4510-8527-7b7a56f89f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:11.514154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b2329-5d33-4510-8527-7b7a56f89f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.514885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b2329-5d33-4510-8527-7b7a56f89f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:11.516485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b2329-5d33-4510-8527-7b7a56f89f7a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:11.532179 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1093/2199] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:11.536580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571b2ee-7daa-40bd-aff7-fa3741bdcfb9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:11.548407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571b2ee-7daa-40bd-aff7-fa3741bdcfb9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:11.554590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f1ebc-ab11-49ba-8125-08edad8f93c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:11.555476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f1ebc-ab11-49ba-8125-08edad8f93c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:11.555755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f1ebc-ab11-49ba-8125-08edad8f93c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:11.556120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f1ebc-ab11-49ba-8125-08edad8f93c5 admin admin] PATH_INFO: `/v3/projects/efe81245e8654a5d80332e40f9842ee0/users/f1a9583b8a824b0ba7148cca920c7c3c/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.558574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f1ebc-ab11-49ba-8125-08edad8f93c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe81245e8654a5d80332e40f9842ee0, user_id=f1a9583b8a824b0ba7148cca920c7c3c, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:11.561264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f1ebc-ab11-49ba-8125-08edad8f93c5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:11.572755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1107/2200] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:11 2022] PUT /identity/v3/projects/efe81245e8654a5d80332e40f9842ee0/users/f1a9583b8a824b0ba7148cca920c7c3c/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:11.583185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62650b2-93d6-491a-89f7-464194e4c745 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:11.583477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62650b2-93d6-491a-89f7-464194e4c745 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:11.583753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62650b2-93d6-491a-89f7-464194e4c745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.614618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b62650b2-93d6-491a-89f7-464194e4c745 None None] MFA Rules not processed for user `f1a9583b8a824b0ba7148cca920c7c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:11.616094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b62650b2-93d6-491a-89f7-464194e4c745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:11.699173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1094/2201] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:11 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:11.713053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f1ebc-ab11-49ba-8125-08edad8f93c5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:11.770905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f1ebc-ab11-49ba-8125-08edad8f93c5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:11.774881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c209b-2a79-495b-8d0e-b48fa9748cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:11.775629 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c209b-2a79-495b-8d0e-b48fa9748cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:11.776202 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c209b-2a79-495b-8d0e-b48fa9748cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:11.776707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c209b-2a79-495b-8d0e-b48fa9748cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.788209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c209b-2a79-495b-8d0e-b48fa9748cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:11.789707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c209b-2a79-495b-8d0e-b48fa9748cd9 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:11.800364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1108/2202] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:11 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:11.849259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62650b2-93d6-491a-89f7-464194e4c745 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:11.851280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b62650b2-93d6-491a-89f7-464194e4c745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:11.869669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62650b2-93d6-491a-89f7-464194e4c745 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:11.873294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6c2c8-1e3d-4328-b791-f6df44d74b41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:11.873942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6c2c8-1e3d-4328-b791-f6df44d74b41 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:11.874223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6c2c8-1e3d-4328-b791-f6df44d74b41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:11.874523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6c2c8-1e3d-4328-b791-f6df44d74b41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:11.905119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:11.942564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6c2c8-1e3d-4328-b791-f6df44d74b41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:11.944056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6c2c8-1e3d-4328-b791-f6df44d74b41 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:11.953317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1095/2203] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:11 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:12.576866 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c209b-2a79-495b-8d0e-b48fa9748cd9 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:12.589913 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c209b-2a79-495b-8d0e-b48fa9748cd9 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:12.593920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e04c9d-9ff8-4200-878e-fc2910537e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:12.594523 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04c9d-9ff8-4200-878e-fc2910537e9e service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:12.594826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04c9d-9ff8-4200-878e-fc2910537e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:12.595286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04c9d-9ff8-4200-878e-fc2910537e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:12.597396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b7e04c9d-9ff8-4200-878e-fc2910537e9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:12.623857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:12.662393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e04c9d-9ff8-4200-878e-fc2910537e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:12.663847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e04c9d-9ff8-4200-878e-fc2910537e9e service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:12.673198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1109/2204] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:13.248833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6c2c8-1e3d-4328-b791-f6df44d74b41 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:13.257783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6c2c8-1e3d-4328-b791-f6df44d74b41 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:13.262754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93410c-5588-42b8-8e52-2eb1f188dbf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:13.263412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93410c-5588-42b8-8e52-2eb1f188dbf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:13.263848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93410c-5588-42b8-8e52-2eb1f188dbf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:13.264135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93410c-5588-42b8-8e52-2eb1f188dbf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:13.336811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93410c-5588-42b8-8e52-2eb1f188dbf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:13.338380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93410c-5588-42b8-8e52-2eb1f188dbf6 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:13.349220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1096/2205] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:15.382759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e04c9d-9ff8-4200-878e-fc2910537e9e service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:15.445067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e04c9d-9ff8-4200-878e-fc2910537e9e service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:15.450275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:15.451167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:15.451443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:15.451721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] PATH_INFO: `/v3/users/acc80d06d3684b588b4322a34dd70f86` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:15.452993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc80d06d3684b588b4322a34dd70f86)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:15.454931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:15.504059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc80d06d3684b588b4322a34dd70f86'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:15.516058 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc80d06d3684b588b4322a34dd70f86'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:15.524900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc80d06d3684b588b4322a34dd70f86'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:15.525563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc80d06d3684b588b4322a34dd70f86'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:15.532828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc80d06d3684b588b4322a34dd70f86'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:15.539018 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1110/2206] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:15 2022] DELETE /identity/v3/users/acc80d06d3684b588b4322a34dd70f86 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:15.543766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac0575-8133-45bf-be1f-66209bddc484 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:15.544290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac0575-8133-45bf-be1f-66209bddc484 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:15.544396 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac0575-8133-45bf-be1f-66209bddc484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:15.561943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e6ac0575-8133-45bf-be1f-66209bddc484 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:15.563139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e6ac0575-8133-45bf-be1f-66209bddc484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:15.627139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1097/2207] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:15.633757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:15.635459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:15.685014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:15.698170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16d498-de02-4ba7-afb7-4e91daeb3ec7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:15.702327 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e37db0-e351-43f7-8314-0f114fc6be29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:15.703049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e37db0-e351-43f7-8314-0f114fc6be29 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:15.703419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e37db0-e351-43f7-8314-0f114fc6be29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:15.703698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e37db0-e351-43f7-8314-0f114fc6be29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:15.713316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e37db0-e351-43f7-8314-0f114fc6be29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:15.714755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e37db0-e351-43f7-8314-0f114fc6be29 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:15.724724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1111/2208] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:16.054488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac0575-8133-45bf-be1f-66209bddc484 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:16.056279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e6ac0575-8133-45bf-be1f-66209bddc484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:16.069989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac0575-8133-45bf-be1f-66209bddc484 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:16.080009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:16.081154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:16.081530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:16.081898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] PATH_INFO: `/v3/projects/f36a54ecd3744e18ab8bca58bfe9e0d0` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:16.084072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36a54ecd3744e18ab8bca58bfe9e0d0)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:16.087478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:16.142741 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36a54ecd3744e18ab8bca58bfe9e0d0', 'request_id': 'req-d6e3a984-ce68-4489-ad00-950fb2d5e839'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.150700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36a54ecd3744e18ab8bca58bfe9e0d0', 'request_id': 'req-d6e3a984-ce68-4489-ad00-950fb2d5e839'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.154327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:16.161206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36a54ecd3744e18ab8bca58bfe9e0d0', 'request_id': 'req-d6e3a984-ce68-4489-ad00-950fb2d5e839'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.174027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] The token cache is being invalidate because project f36a54ecd3744e18ab8bca58bfe9e0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:16.174314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.174978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.176796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1098/2209] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:16 2022] DELETE /identity/v3/projects/f36a54ecd3744e18ab8bca58bfe9e0d0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:16.180972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e37db0-e351-43f7-8314-0f114fc6be29 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:16.193117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e37db0-e351-43f7-8314-0f114fc6be29 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:16.271313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:16.272680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:16.272909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:16.273169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] PATH_INFO: `/v3/users/f1a9583b8a824b0ba7148cca920c7c3c` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:16.274541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a9583b8a824b0ba7148cca920c7c3c)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:16.277206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:16.349442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a9583b8a824b0ba7148cca920c7c3c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.364753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a9583b8a824b0ba7148cca920c7c3c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.375785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a9583b8a824b0ba7148cca920c7c3c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.376462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a9583b8a824b0ba7148cca920c7c3c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.386571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b846391-2e69-41e0-b48b-ee46639c0aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a9583b8a824b0ba7148cca920c7c3c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.396491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1112/2210] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:16 2022] DELETE /identity/v3/users/f1a9583b8a824b0ba7148cca920c7c3c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:16.723619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:16.725270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:16.741734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3a984-ce68-4489-ad00-950fb2d5e839 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:16.805459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:16.806365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:16.806668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:16.806933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] PATH_INFO: `/v3/projects/efe81245e8654a5d80332e40f9842ee0` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:16.808130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe81245e8654a5d80332e40f9842ee0)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:16.810072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:16.859000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe81245e8654a5d80332e40f9842ee0', 'request_id': 'req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.865156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe81245e8654a5d80332e40f9842ee0', 'request_id': 'req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.876494 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe81245e8654a5d80332e40f9842ee0', 'request_id': 'req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.894976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] The token cache is being invalidate because project efe81245e8654a5d80332e40f9842ee0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:16.895271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.895949 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7bd78916-4cec-4d68-875b-d6e7f4e23dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:16.897881 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1099/2211] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:16 2022] DELETE /identity/v3/projects/efe81245e8654a5d80332e40f9842ee0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:16.903009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41318e-b65d-4672-bd14-0a2614b6949d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:16.903263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41318e-b65d-4672-bd14-0a2614b6949d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:16.903571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41318e-b65d-4672-bd14-0a2614b6949d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:16.921660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1a41318e-b65d-4672-bd14-0a2614b6949d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:16.923320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1a41318e-b65d-4672-bd14-0a2614b6949d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:16.996400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1113/2212] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:17.004051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922444b-7a5a-41ac-9b28-3eb38463f5ab None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.004457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922444b-7a5a-41ac-9b28-3eb38463f5ab None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.004837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922444b-7a5a-41ac-9b28-3eb38463f5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.024206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0922444b-7a5a-41ac-9b28-3eb38463f5ab None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:17.025777 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0922444b-7a5a-41ac-9b28-3eb38463f5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:17.034246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1100/2213] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:17.038530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41318e-b65d-4672-bd14-0a2614b6949d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:17.048301 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41318e-b65d-4672-bd14-0a2614b6949d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:17.054171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c133e1-9257-439a-b18e-ecc7b8f048dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:17.054827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c133e1-9257-439a-b18e-ecc7b8f048dc admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.055098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c133e1-9257-439a-b18e-ecc7b8f048dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.055374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c133e1-9257-439a-b18e-ecc7b8f048dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.056115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c133e1-9257-439a-b18e-ecc7b8f048dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:37:17.056424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c133e1-9257-439a-b18e-ecc7b8f048dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:17.058263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c133e1-9257-439a-b18e-ecc7b8f048dc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:17.059926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:17.072895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1114/2214] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:37:17.076082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922444b-7a5a-41ac-9b28-3eb38463f5ab None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:17.085919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922444b-7a5a-41ac-9b28-3eb38463f5ab None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:17.091707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15131c9-3886-4ea5-8253-b60eed19d796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:17.092395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15131c9-3886-4ea5-8253-b60eed19d796 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.092678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15131c9-3886-4ea5-8253-b60eed19d796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.092975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15131c9-3886-4ea5-8253-b60eed19d796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.093686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15131c9-3886-4ea5-8253-b60eed19d796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:17.095626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15131c9-3886-4ea5-8253-b60eed19d796 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:17.125758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1101/2215] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:17 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:17.126964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c133e1-9257-439a-b18e-ecc7b8f048dc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:17.137491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c133e1-9257-439a-b18e-ecc7b8f048dc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:17.196314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37ba2e-a3c3-4d31-a8d3-fc39c7e78527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:17.197027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37ba2e-a3c3-4d31-a8d3-fc39c7e78527 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.197334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37ba2e-a3c3-4d31-a8d3-fc39c7e78527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.197611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37ba2e-a3c3-4d31-a8d3-fc39c7e78527 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.198332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37ba2e-a3c3-4d31-a8d3-fc39c7e78527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:17.200252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37ba2e-a3c3-4d31-a8d3-fc39c7e78527 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:17.224549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1115/2216] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:17 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:17.225925 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15131c9-3886-4ea5-8253-b60eed19d796 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:17.246465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15131c9-3886-4ea5-8253-b60eed19d796 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:17.251378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02581c19-65c4-4d8b-a859-e95800d5b826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:17.252054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02581c19-65c4-4d8b-a859-e95800d5b826 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.252339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02581c19-65c4-4d8b-a859-e95800d5b826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.252621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02581c19-65c4-4d8b-a859-e95800d5b826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.253936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02581c19-65c4-4d8b-a859-e95800d5b826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:17.255599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02581c19-65c4-4d8b-a859-e95800d5b826 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:17.268010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1102/2217] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:17.271973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37ba2e-a3c3-4d31-a8d3-fc39c7e78527 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:17.283477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37ba2e-a3c3-4d31-a8d3-fc39c7e78527 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:17.289314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f403f2-cc42-4f1d-8152-cf2fa03ef5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:17.290352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f403f2-cc42-4f1d-8152-cf2fa03ef5af admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.290352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f403f2-cc42-4f1d-8152-cf2fa03ef5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.290571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f403f2-cc42-4f1d-8152-cf2fa03ef5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.291350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f403f2-cc42-4f1d-8152-cf2fa03ef5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:17.293383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f403f2-cc42-4f1d-8152-cf2fa03ef5af admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:17.304779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1116/2218] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:37:17.309978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02581c19-65c4-4d8b-a859-e95800d5b826 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:17.320146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02581c19-65c4-4d8b-a859-e95800d5b826 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:17.325643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acb48f-aea3-4c58-b5c6-143769978b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:17.326594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acb48f-aea3-4c58-b5c6-143769978b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.326849 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acb48f-aea3-4c58-b5c6-143769978b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.327118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acb48f-aea3-4c58-b5c6-143769978b21 admin admin] PATH_INFO: `/v3/projects/64524e73ca0f405cbccd891aa3564988/users/9c12d080b671442580f33071a0a8efab/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.343819 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acb48f-aea3-4c58-b5c6-143769978b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64524e73ca0f405cbccd891aa3564988, user_id=9c12d080b671442580f33071a0a8efab, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:17.346664 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acb48f-aea3-4c58-b5c6-143769978b21 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:17.375857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1103/2219] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:17 2022] PUT /identity/v3/projects/64524e73ca0f405cbccd891aa3564988/users/9c12d080b671442580f33071a0a8efab/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:17.383291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48816de0-d619-4587-b254-c28e74441c26 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.383291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48816de0-d619-4587-b254-c28e74441c26 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.383291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48816de0-d619-4587-b254-c28e74441c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.401441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-48816de0-d619-4587-b254-c28e74441c26 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:17.402681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-48816de0-d619-4587-b254-c28e74441c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:17.408144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acb48f-aea3-4c58-b5c6-143769978b21 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:17.410918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c9acb48f-aea3-4c58-b5c6-143769978b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:17.427068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:17.483887 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1117/2220] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:17.493918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48816de0-d619-4587-b254-c28e74441c26 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:17.495727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-48816de0-d619-4587-b254-c28e74441c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:17.530490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acb48f-aea3-4c58-b5c6-143769978b21 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:17.535345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:17.536254 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.536533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.536839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] PATH_INFO: `/v3/users/fcd289bcf55340f6b80b62bf3dc7d513` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.538053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcd289bcf55340f6b80b62bf3dc7d513)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:17.540071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:17.588691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48816de0-d619-4587-b254-c28e74441c26 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:17.593167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7157d2-0046-4610-8ece-69b6eabdaa13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:17.593821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7157d2-0046-4610-8ece-69b6eabdaa13 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.594111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7157d2-0046-4610-8ece-69b6eabdaa13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.594377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7157d2-0046-4610-8ece-69b6eabdaa13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.602596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcd289bcf55340f6b80b62bf3dc7d513'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:17.605158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7157d2-0046-4610-8ece-69b6eabdaa13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:17.606841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7157d2-0046-4610-8ece-69b6eabdaa13 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:17.616300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcd289bcf55340f6b80b62bf3dc7d513'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:17.620302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1118/2221] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:17.626993 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcd289bcf55340f6b80b62bf3dc7d513'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:17.638591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcd289bcf55340f6b80b62bf3dc7d513'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:17.639364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcd289bcf55340f6b80b62bf3dc7d513'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:17.648823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1104/2222] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:17 2022] DELETE /identity/v3/users/fcd289bcf55340f6b80b62bf3dc7d513 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:17.651811 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b23b-bdea-4a66-892b-0cb9576e21dd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.652144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b23b-bdea-4a66-892b-0cb9576e21dd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.652408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b23b-bdea-4a66-892b-0cb9576e21dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.670256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c3a3b23b-bdea-4a66-892b-0cb9576e21dd None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:17.671478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c3a3b23b-bdea-4a66-892b-0cb9576e21dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:17.681209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:17.737115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1119/2223] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:17.748694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:17.750237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:17.814035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e30c7f-c8f8-4f5b-a166-6b0ae20f0565 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:17.817886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc04c8-bfa2-41a8-8cd4-cc4b4dad13fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:17.818537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc04c8-bfa2-41a8-8cd4-cc4b4dad13fe service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:17.818850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc04c8-bfa2-41a8-8cd4-cc4b4dad13fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:17.819127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc04c8-bfa2-41a8-8cd4-cc4b4dad13fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:17.829382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc04c8-bfa2-41a8-8cd4-cc4b4dad13fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:17.830809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc04c8-bfa2-41a8-8cd4-cc4b4dad13fe service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:17.841203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1105/2224] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:18.212447 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b23b-bdea-4a66-892b-0cb9576e21dd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:18.214778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c3a3b23b-bdea-4a66-892b-0cb9576e21dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:18.228998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b23b-bdea-4a66-892b-0cb9576e21dd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:18.234562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:18.235681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:18.236782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:18.236782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] PATH_INFO: `/v3/projects/7c8f7705b24b412e903df811fee2320e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:18.237713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c8f7705b24b412e903df811fee2320e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:18.239694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:18.296219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c8f7705b24b412e903df811fee2320e', 'request_id': 'req-fb0453fe-468b-435c-93d9-bebcd3db8ff7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.305413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c8f7705b24b412e903df811fee2320e', 'request_id': 'req-fb0453fe-468b-435c-93d9-bebcd3db8ff7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.312195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c8f7705b24b412e903df811fee2320e', 'request_id': 'req-fb0453fe-468b-435c-93d9-bebcd3db8ff7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.324251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] The token cache is being invalidate because project 7c8f7705b24b412e903df811fee2320e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:18.324479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.325155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb0453fe-468b-435c-93d9-bebcd3db8ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.327020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1120/2225] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:18 2022] DELETE /identity/v3/projects/7c8f7705b24b412e903df811fee2320e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:18.330918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc04c8-bfa2-41a8-8cd4-cc4b4dad13fe service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:18.341834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc04c8-bfa2-41a8-8cd4-cc4b4dad13fe service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:18.379165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:18.405363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73d4c5-ebd4-42c7-8e79-521087df8163 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:18.405913 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73d4c5-ebd4-42c7-8e79-521087df8163 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:18.405913 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73d4c5-ebd4-42c7-8e79-521087df8163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:18.412526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:18.413736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:18.414079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:18.414457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] PATH_INFO: `/v3/users/8af0c8c971ea4b339c0b74dd071131f8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:18.416330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af0c8c971ea4b339c0b74dd071131f8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:18.418456 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:18.445522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6b73d4c5-ebd4-42c7-8e79-521087df8163 None None] MFA Rules not processed for user `9c12d080b671442580f33071a0a8efab`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:18.447951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6b73d4c5-ebd4-42c7-8e79-521087df8163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:18.485784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af0c8c971ea4b339c0b74dd071131f8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.502267 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af0c8c971ea4b339c0b74dd071131f8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.515394 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af0c8c971ea4b339c0b74dd071131f8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.526062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af0c8c971ea4b339c0b74dd071131f8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.526815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af0c8c971ea4b339c0b74dd071131f8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.535722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1106/2226] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:18 2022] DELETE /identity/v3/users/8af0c8c971ea4b339c0b74dd071131f8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:18.546119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1121/2227] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:37:18.554284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:18.563199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:18.713408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c9723-8880-440f-9398-1df80e5ba07e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:18.719449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb2dfb-0c34-4362-abc0-06e4bf20c03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:18.720139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb2dfb-0c34-4362-abc0-06e4bf20c03d admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:18.720427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb2dfb-0c34-4362-abc0-06e4bf20c03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:18.720705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb2dfb-0c34-4362-abc0-06e4bf20c03d admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:18.721452 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb2dfb-0c34-4362-abc0-06e4bf20c03d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:18.723373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb2dfb-0c34-4362-abc0-06e4bf20c03d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:18.764176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1107/2228] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:18 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:18.766428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73d4c5-ebd4-42c7-8e79-521087df8163 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:18.778320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73d4c5-ebd4-42c7-8e79-521087df8163 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:18.809672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:18.844189 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd12fe-bf0a-4551-a0b1-833095f5a3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:18.844675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd12fe-bf0a-4551-a0b1-833095f5a3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:18.844984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd12fe-bf0a-4551-a0b1-833095f5a3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:18.845269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd12fe-bf0a-4551-a0b1-833095f5a3b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:18.847595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd12fe-bf0a-4551-a0b1-833095f5a3b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:18.847959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd12fe-bf0a-4551-a0b1-833095f5a3b3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:18.871695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1122/2229] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:18 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:37:18.876686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb2dfb-0c34-4362-abc0-06e4bf20c03d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:18.887105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb2dfb-0c34-4362-abc0-06e4bf20c03d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:18.892205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e1c3e-2a50-4ffa-9bfc-6e183c77260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:18.892910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e1c3e-2a50-4ffa-9bfc-6e183c77260e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:18.893193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e1c3e-2a50-4ffa-9bfc-6e183c77260e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:18.893468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e1c3e-2a50-4ffa-9bfc-6e183c77260e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:18.894200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e1c3e-2a50-4ffa-9bfc-6e183c77260e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:18.895873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e1c3e-2a50-4ffa-9bfc-6e183c77260e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:18.898461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd12fe-bf0a-4551-a0b1-833095f5a3b3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:18.900585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-facd12fe-bf0a-4551-a0b1-833095f5a3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:18.907122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1108/2230] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:18.911393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e1c3e-2a50-4ffa-9bfc-6e183c77260e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:18.916452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd12fe-bf0a-4551-a0b1-833095f5a3b3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:18.921649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:18.922510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e1c3e-2a50-4ffa-9bfc-6e183c77260e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:18.922627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:18.922838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:18.923162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] PATH_INFO: `/v3/projects/5bebca6bf6c541cfa60bb506c26d0b44` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:18.924388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bebca6bf6c541cfa60bb506c26d0b44)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:18.926396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:18.927366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a690e-5b0b-4cc0-bd17-90b86e5a8ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:18.928412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a690e-5b0b-4cc0-bd17-90b86e5a8ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:18.928693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a690e-5b0b-4cc0-bd17-90b86e5a8ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:18.929147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a690e-5b0b-4cc0-bd17-90b86e5a8ac4 admin admin] PATH_INFO: `/v3/projects/e66fea7a5cd440a58485e2884225d236/users/c3fc03513df6437ba52b3fbee7175e87/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:18.945352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a690e-5b0b-4cc0-bd17-90b86e5a8ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66fea7a5cd440a58485e2884225d236, user_id=c3fc03513df6437ba52b3fbee7175e87, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:18.948143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a690e-5b0b-4cc0-bd17-90b86e5a8ac4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:18.963516 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1109/2231] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:18 2022] PUT /identity/v3/projects/e66fea7a5cd440a58485e2884225d236/users/c3fc03513df6437ba52b3fbee7175e87/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:18.967785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a690e-5b0b-4cc0-bd17-90b86e5a8ac4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:18.972621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:18.978226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bebca6bf6c541cfa60bb506c26d0b44', 'request_id': 'req-267c7422-9103-4589-b71d-b3468d4ad726'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.986642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bebca6bf6c541cfa60bb506c26d0b44', 'request_id': 'req-267c7422-9103-4589-b71d-b3468d4ad726'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:18.992381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bebca6bf6c541cfa60bb506c26d0b44', 'request_id': 'req-267c7422-9103-4589-b71d-b3468d4ad726'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.005553 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] The token cache is being invalidate because project 5bebca6bf6c541cfa60bb506c26d0b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:19.005839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.006491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.008035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1123/2232] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:18 2022] DELETE /identity/v3/projects/5bebca6bf6c541cfa60bb506c26d0b44 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:19.012188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:19.013669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:19.033701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a690e-5b0b-4cc0-bd17-90b86e5a8ac4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:19.078621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c7422-9103-4589-b71d-b3468d4ad726 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:19.083683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:19.084572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:19.084887 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:19.085169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] PATH_INFO: `/v3/users/fc24abe470894bf78b11dfe31b1d5296` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:19.086366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc24abe470894bf78b11dfe31b1d5296)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:19.088853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e375558-d42b-4779-a438-e6b28db4653d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:19.089073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:19.089535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e375558-d42b-4779-a438-e6b28db4653d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:19.089819 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e375558-d42b-4779-a438-e6b28db4653d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:19.090153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e375558-d42b-4779-a438-e6b28db4653d admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:19.091182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e375558-d42b-4779-a438-e6b28db4653d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:19.092797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e375558-d42b-4779-a438-e6b28db4653d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:19.103515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1110/2233] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:37:19.108051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e375558-d42b-4779-a438-e6b28db4653d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:19.109841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8e375558-d42b-4779-a438-e6b28db4653d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:19.122122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e375558-d42b-4779-a438-e6b28db4653d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:19.128685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029335dc-b75c-4158-9f8e-a1c04748bcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:19.129627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029335dc-b75c-4158-9f8e-a1c04748bcce admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:19.129915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029335dc-b75c-4158-9f8e-a1c04748bcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:19.130193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029335dc-b75c-4158-9f8e-a1c04748bcce admin admin] PATH_INFO: `/v3/projects/e66fea7a5cd440a58485e2884225d236/users/c3fc03513df6437ba52b3fbee7175e87/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:19.133512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029335dc-b75c-4158-9f8e-a1c04748bcce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66fea7a5cd440a58485e2884225d236, user_id=c3fc03513df6437ba52b3fbee7175e87, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:19.136850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029335dc-b75c-4158-9f8e-a1c04748bcce admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:19.137620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=83,keystone:INSERT=4,keystone:DELETE=13 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:19.140606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc24abe470894bf78b11dfe31b1d5296'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.151445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1111/2234] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:19 2022] PUT /identity/v3/projects/e66fea7a5cd440a58485e2884225d236/users/c3fc03513df6437ba52b3fbee7175e87/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:19.157951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc24abe470894bf78b11dfe31b1d5296'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.167143 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc24abe470894bf78b11dfe31b1d5296'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.168035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc24abe470894bf78b11dfe31b1d5296'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.175320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3b8c4d2-78dd-49f1-80dc-af991da8c4c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc24abe470894bf78b11dfe31b1d5296'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.181530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1124/2235] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:19 2022] DELETE /identity/v3/users/fc24abe470894bf78b11dfe31b1d5296 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:19.488695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029335dc-b75c-4158-9f8e-a1c04748bcce admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:19.491043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-029335dc-b75c-4158-9f8e-a1c04748bcce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:19.626944 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029335dc-b75c-4158-9f8e-a1c04748bcce admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:19.687881 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:19.736542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:19.738061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:19.738350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:19.738902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] PATH_INFO: `/v3/projects/bc67e022e8ae4cceb3da72e12bf8899e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:19.741272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc67e022e8ae4cceb3da72e12bf8899e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:19.745334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:19.827024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc67e022e8ae4cceb3da72e12bf8899e', 'request_id': 'req-45442bf2-a5af-4c69-9933-c3b53c135c76'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.833987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc67e022e8ae4cceb3da72e12bf8899e', 'request_id': 'req-45442bf2-a5af-4c69-9933-c3b53c135c76'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.846045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc67e022e8ae4cceb3da72e12bf8899e', 'request_id': 'req-45442bf2-a5af-4c69-9933-c3b53c135c76'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.860661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] The token cache is being invalidate because project bc67e022e8ae4cceb3da72e12bf8899e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:19.861082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.862219 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:19.872850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1112/2236] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:19 2022] DELETE /identity/v3/projects/bc67e022e8ae4cceb3da72e12bf8899e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:20.092955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9330d02-18f2-4c7c-9321-be63e6e1d9d8 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:20.092955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9330d02-18f2-4c7c-9321-be63e6e1d9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:20.092955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9330d02-18f2-4c7c-9321-be63e6e1d9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:20.133890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d9330d02-18f2-4c7c-9321-be63e6e1d9d8 None None] MFA Rules not processed for user `c3fc03513df6437ba52b3fbee7175e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:20.135139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d9330d02-18f2-4c7c-9321-be63e6e1d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:20.251556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1125/2237] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 24 10:37:20.269055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:20.270695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:20.357957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45442bf2-a5af-4c69-9933-c3b53c135c76 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:20.362734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e1132-2261-4b7a-9d64-efa664542e01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:20.363405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e1132-2261-4b7a-9d64-efa664542e01 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:20.363853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e1132-2261-4b7a-9d64-efa664542e01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:20.364165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e1132-2261-4b7a-9d64-efa664542e01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:20.366730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1a1e1132-2261-4b7a-9d64-efa664542e01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:20.453828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e1132-2261-4b7a-9d64-efa664542e01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:20.455488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e1132-2261-4b7a-9d64-efa664542e01 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:20.466233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1113/2238] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:21.837013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9330d02-18f2-4c7c-9321-be63e6e1d9d8 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:21.838913 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d9330d02-18f2-4c7c-9321-be63e6e1d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:21.853229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9330d02-18f2-4c7c-9321-be63e6e1d9d8 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:21.858321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892753b9-91e4-443e-b5e6-8b361f6d4aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:21.858976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892753b9-91e4-443e-b5e6-8b361f6d4aec service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:21.859259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892753b9-91e4-443e-b5e6-8b361f6d4aec service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:21.859532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892753b9-91e4-443e-b5e6-8b361f6d4aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:21.870493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892753b9-91e4-443e-b5e6-8b361f6d4aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:21.873250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892753b9-91e4-443e-b5e6-8b361f6d4aec service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:21.891890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1126/2239] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:21 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:25.083800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c38a1-138d-4cf7-b90f-8f1d14145e4d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:25.084285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c38a1-138d-4cf7-b90f-8f1d14145e4d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:25.084285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c38a1-138d-4cf7-b90f-8f1d14145e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:25.103058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-531c38a1-138d-4cf7-b90f-8f1d14145e4d None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:25.104293 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-531c38a1-138d-4cf7-b90f-8f1d14145e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:25.125004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:25.165400 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1114/2240] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:37:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:25.215520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892753b9-91e4-443e-b5e6-8b361f6d4aec service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:25.217578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-892753b9-91e4-443e-b5e6-8b361f6d4aec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:25.311699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892753b9-91e4-443e-b5e6-8b361f6d4aec service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:25.317251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab83fb7-fc7a-48b7-a318-ed02de45e275 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:25.317985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab83fb7-fc7a-48b7-a318-ed02de45e275 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:25.318337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab83fb7-fc7a-48b7-a318-ed02de45e275 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:25.318685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab83fb7-fc7a-48b7-a318-ed02de45e275 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:25.328440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab83fb7-fc7a-48b7-a318-ed02de45e275 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:25.330208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab83fb7-fc7a-48b7-a318-ed02de45e275 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:25.340450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1127/2241] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:37:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 10:37:25.880464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff49a3-b02f-4943-8168-8faa230cbeb0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:25.881044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff49a3-b02f-4943-8168-8faa230cbeb0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:25.881582 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff49a3-b02f-4943-8168-8faa230cbeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:25.909186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-08ff49a3-b02f-4943-8168-8faa230cbeb0 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:25.910597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-08ff49a3-b02f-4943-8168-8faa230cbeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:25.919596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1115/2242] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:37:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:25.946462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab83fb7-fc7a-48b7-a318-ed02de45e275 service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:25.961642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab83fb7-fc7a-48b7-a318-ed02de45e275 service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:25.969134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469083-f5ca-4762-beb0-e7608c33b629 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:25.969965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469083-f5ca-4762-beb0-e7608c33b629 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:25.970546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469083-f5ca-4762-beb0-e7608c33b629 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:25.971025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469083-f5ca-4762-beb0-e7608c33b629 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:25.987059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d469083-f5ca-4762-beb0-e7608c33b629 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:25.990685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d469083-f5ca-4762-beb0-e7608c33b629 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:26.006537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1128/2243] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:37:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:37:26.864326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4544935-64c7-4584-add9-39e3ce2faa51 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:26.864866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4544935-64c7-4584-add9-39e3ce2faa51 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:26.865123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4544935-64c7-4584-add9-39e3ce2faa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:26.890017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d4544935-64c7-4584-add9-39e3ce2faa51 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:26.891671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d4544935-64c7-4584-add9-39e3ce2faa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:26.902219 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1116/2244] 10.222.0.69 () {58 vars in 967 bytes} [Fri Jun 24 10:37:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:26.926783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469083-f5ca-4762-beb0-e7608c33b629 service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:26.941435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469083-f5ca-4762-beb0-e7608c33b629 service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:26.974429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47230-4710-4c62-a236-3ad7871f16fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:26.976399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47230-4710-4c62-a236-3ad7871f16fa service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:26.977608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47230-4710-4c62-a236-3ad7871f16fa service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:26.977964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47230-4710-4c62-a236-3ad7871f16fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:27.019688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:27.023942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a47230-4710-4c62-a236-3ad7871f16fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:27.026650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a47230-4710-4c62-a236-3ad7871f16fa service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:27.045561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1129/2245] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:37:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:37:27.424024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693307af-c2f5-4ca9-ba1a-6cb21b2457b4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:27.424339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693307af-c2f5-4ca9-ba1a-6cb21b2457b4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:27.424615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693307af-c2f5-4ca9-ba1a-6cb21b2457b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:27.450598 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-693307af-c2f5-4ca9-ba1a-6cb21b2457b4 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:27.453245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-693307af-c2f5-4ca9-ba1a-6cb21b2457b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:27.468827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1117/2246] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:27.486497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47230-4710-4c62-a236-3ad7871f16fa service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:27.500250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47230-4710-4c62-a236-3ad7871f16fa service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:27.505677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b30d88-4275-487a-b742-8a23a63d2b80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:27.506539 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b30d88-4275-487a-b742-8a23a63d2b80 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:27.506979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b30d88-4275-487a-b742-8a23a63d2b80 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:27.507401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b30d88-4275-487a-b742-8a23a63d2b80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:27.517099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b30d88-4275-487a-b742-8a23a63d2b80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:27.519630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b30d88-4275-487a-b742-8a23a63d2b80 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:27.530753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1130/2247] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:37:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:37:27.942030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66538ec0-fbdc-4e04-a864-692fc488c1f5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:27.943757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66538ec0-fbdc-4e04-a864-692fc488c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:27.944259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66538ec0-fbdc-4e04-a864-692fc488c1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:27.967307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-66538ec0-fbdc-4e04-a864-692fc488c1f5 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:27.968877 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-66538ec0-fbdc-4e04-a864-692fc488c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:27.983594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1118/2248] 10.222.0.69 () {58 vars in 967 bytes} [Fri Jun 24 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:27.994061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b30d88-4275-487a-b742-8a23a63d2b80 service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:28.013886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b30d88-4275-487a-b742-8a23a63d2b80 service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:28.023285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33153491-45ab-4c47-a511-b687fd055088 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:28.024131 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33153491-45ab-4c47-a511-b687fd055088 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:28.024460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33153491-45ab-4c47-a511-b687fd055088 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:28.024825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33153491-45ab-4c47-a511-b687fd055088 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:28.036916 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33153491-45ab-4c47-a511-b687fd055088 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:28.038877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33153491-45ab-4c47-a511-b687fd055088 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:28.053205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1131/2249] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:37:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 10:37:30.684679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66538ec0-fbdc-4e04-a864-692fc488c1f5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:30.687306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-66538ec0-fbdc-4e04-a864-692fc488c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:30.829672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66538ec0-fbdc-4e04-a864-692fc488c1f5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:30.844242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:30.845704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:30.846329 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:30.846768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] PATH_INFO: `/v3/users/57cee90ceb214b68b07ef3f8ff0b02db` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:30.848619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57cee90ceb214b68b07ef3f8ff0b02db)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:30.851583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:30.948869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57cee90ceb214b68b07ef3f8ff0b02db'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:30.979040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57cee90ceb214b68b07ef3f8ff0b02db'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:30.995872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57cee90ceb214b68b07ef3f8ff0b02db'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:31.019236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57cee90ceb214b68b07ef3f8ff0b02db'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:31.020692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57cee90ceb214b68b07ef3f8ff0b02db'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:31.030246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1119/2250] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:30 2022] DELETE /identity/v3/users/57cee90ceb214b68b07ef3f8ff0b02db => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:31.034552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096b757-1152-4174-8119-df9584cb035c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:31.041627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096b757-1152-4174-8119-df9584cb035c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:31.042602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096b757-1152-4174-8119-df9584cb035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:31.107033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d096b757-1152-4174-8119-df9584cb035c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:31.112618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d096b757-1152-4174-8119-df9584cb035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:31.292334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1132/2251] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:31.303446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:31.305425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:31.350593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:31.463161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608433cf-24db-4561-b9b1-6881f610f704 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:31.468102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a764c-e763-4625-b0bb-f4a7c3435ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:31.468888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a764c-e763-4625-b0bb-f4a7c3435ded service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:31.469233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a764c-e763-4625-b0bb-f4a7c3435ded service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:31.469576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a764c-e763-4625-b0bb-f4a7c3435ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:31.482684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a764c-e763-4625-b0bb-f4a7c3435ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:31.484480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a764c-e763-4625-b0bb-f4a7c3435ded service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:31.508533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1120/2252] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:31.775385 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096b757-1152-4174-8119-df9584cb035c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:31.777166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d096b757-1152-4174-8119-df9584cb035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:31.801759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096b757-1152-4174-8119-df9584cb035c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:31.806966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:31.808048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:31.808421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:31.808918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] PATH_INFO: `/v3/projects/ac68515ea7c64764b0c9f4005158e360` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:31.810306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac68515ea7c64764b0c9f4005158e360)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:31.813413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:31.878080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac68515ea7c64764b0c9f4005158e360', 'request_id': 'req-1a8fd2e5-2da6-4618-885c-ff614fe4b836'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:31.889567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac68515ea7c64764b0c9f4005158e360', 'request_id': 'req-1a8fd2e5-2da6-4618-885c-ff614fe4b836'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:31.897145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac68515ea7c64764b0c9f4005158e360', 'request_id': 'req-1a8fd2e5-2da6-4618-885c-ff614fe4b836'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:31.909055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] The token cache is being invalidate because project ac68515ea7c64764b0c9f4005158e360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:31.909609 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:31.910410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1a8fd2e5-2da6-4618-885c-ff614fe4b836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:31.912120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1133/2253] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:31 2022] DELETE /identity/v3/projects/ac68515ea7c64764b0c9f4005158e360 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:31.918737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5221e0a-7b2a-4a08-9491-2c2199926ea3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:31.919302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5221e0a-7b2a-4a08-9491-2c2199926ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:31.919407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5221e0a-7b2a-4a08-9491-2c2199926ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:31.938303 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c5221e0a-7b2a-4a08-9491-2c2199926ea3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:31.940003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c5221e0a-7b2a-4a08-9491-2c2199926ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:32.039753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1121/2254] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:32.041790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c59c-8f2e-47d5-ae95-d403ecce22b2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:32.042318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c59c-8f2e-47d5-ae95-d403ecce22b2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:32.042539 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c59c-8f2e-47d5-ae95-d403ecce22b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:32.071784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3cb6c59c-8f2e-47d5-ae95-d403ecce22b2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:32.073315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3cb6c59c-8f2e-47d5-ae95-d403ecce22b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:32.086359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1134/2255] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:37:32.087410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5221e0a-7b2a-4a08-9491-2c2199926ea3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:32.098443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5221e0a-7b2a-4a08-9491-2c2199926ea3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:32.103427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20bf7a-0211-4e54-8311-4127dd0a3017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:32.104337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20bf7a-0211-4e54-8311-4127dd0a3017 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:32.104800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20bf7a-0211-4e54-8311-4127dd0a3017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:32.105387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20bf7a-0211-4e54-8311-4127dd0a3017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:32.106395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20bf7a-0211-4e54-8311-4127dd0a3017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:37:32.106863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20bf7a-0211-4e54-8311-4127dd0a3017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:32.108616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20bf7a-0211-4e54-8311-4127dd0a3017 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:32.124214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1122/2256] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:37:32.129005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c59c-8f2e-47d5-ae95-d403ecce22b2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:32.139262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c59c-8f2e-47d5-ae95-d403ecce22b2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:32.144509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bbd7e-d67c-4608-8eec-8861c955a32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:32.145400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bbd7e-d67c-4608-8eec-8861c955a32a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:32.146115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bbd7e-d67c-4608-8eec-8861c955a32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:32.146400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bbd7e-d67c-4608-8eec-8861c955a32a admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:32.147104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bbd7e-d67c-4608-8eec-8861c955a32a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:32.149396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bbd7e-d67c-4608-8eec-8861c955a32a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:32.187340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1135/2257] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:32 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:32.191660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20bf7a-0211-4e54-8311-4127dd0a3017 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:32.202173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20bf7a-0211-4e54-8311-4127dd0a3017 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:32.273535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291347ab-df52-431a-8c0a-be7a47c87754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:32.274247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291347ab-df52-431a-8c0a-be7a47c87754 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:32.274673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291347ab-df52-431a-8c0a-be7a47c87754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:32.275043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291347ab-df52-431a-8c0a-be7a47c87754 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:32.275903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291347ab-df52-431a-8c0a-be7a47c87754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:32.278901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291347ab-df52-431a-8c0a-be7a47c87754 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:32.308903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1123/2258] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:32 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:32.309761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bbd7e-d67c-4608-8eec-8861c955a32a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:32.320704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bbd7e-d67c-4608-8eec-8861c955a32a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:32.327401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1532b0-e665-43e5-8f5d-43e774baafa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:32.328114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1532b0-e665-43e5-8f5d-43e774baafa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:32.328362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1532b0-e665-43e5-8f5d-43e774baafa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:32.328636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1532b0-e665-43e5-8f5d-43e774baafa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:32.329451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1532b0-e665-43e5-8f5d-43e774baafa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:32.331233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1532b0-e665-43e5-8f5d-43e774baafa3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:32.345762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1136/2259] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:37:32.349829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291347ab-df52-431a-8c0a-be7a47c87754 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:32.360420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291347ab-df52-431a-8c0a-be7a47c87754 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:32.367651 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712de8d-d4b4-4ac0-9c00-7f8a4779e26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:32.368332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712de8d-d4b4-4ac0-9c00-7f8a4779e26e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:32.368616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712de8d-d4b4-4ac0-9c00-7f8a4779e26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:32.369009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712de8d-d4b4-4ac0-9c00-7f8a4779e26e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:32.369674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712de8d-d4b4-4ac0-9c00-7f8a4779e26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:32.371333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712de8d-d4b4-4ac0-9c00-7f8a4779e26e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:32.384037 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1124/2260] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:32.388016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1532b0-e665-43e5-8f5d-43e774baafa3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:32.399683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1532b0-e665-43e5-8f5d-43e774baafa3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:32.407242 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44638795-aecd-460e-9019-3add102b345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:32.408642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638795-aecd-460e-9019-3add102b345c admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:32.409224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638795-aecd-460e-9019-3add102b345c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:32.409700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638795-aecd-460e-9019-3add102b345c admin admin] PATH_INFO: `/v3/projects/b88db10e9091496a84e87446da50a3b6/users/60367bf464f4441483bfe4a1373a1e92/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:32.425653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:32.432464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44638795-aecd-460e-9019-3add102b345c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88db10e9091496a84e87446da50a3b6, user_id=60367bf464f4441483bfe4a1373a1e92, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:32.437804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44638795-aecd-460e-9019-3add102b345c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:32.476806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1137/2261] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:32 2022] PUT /identity/v3/projects/b88db10e9091496a84e87446da50a3b6/users/60367bf464f4441483bfe4a1373a1e92/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:32.479434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9297-c332-4565-8f50-f916d00a415b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:32.479735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9297-c332-4565-8f50-f916d00a415b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:32.479999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9297-c332-4565-8f50-f916d00a415b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:32.526112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5b9c9297-c332-4565-8f50-f916d00a415b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:32.529831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5b9c9297-c332-4565-8f50-f916d00a415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:32.597360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:32.649326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1125/2262] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:32.660404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44638795-aecd-460e-9019-3add102b345c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:32.662826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-44638795-aecd-460e-9019-3add102b345c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:32.756034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44638795-aecd-460e-9019-3add102b345c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:32.760636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f105b8-7e80-44cb-8eb0-b272d12ab3c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:32.761875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f105b8-7e80-44cb-8eb0-b272d12ab3c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:32.762218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f105b8-7e80-44cb-8eb0-b272d12ab3c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:32.762753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f105b8-7e80-44cb-8eb0-b272d12ab3c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:32.801383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f105b8-7e80-44cb-8eb0-b272d12ab3c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:32.805826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f105b8-7e80-44cb-8eb0-b272d12ab3c7 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:32.838501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1138/2263] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:33.806777 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ab093-943e-4e45-b1d4-a09aa7dae672 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:33.807308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ab093-943e-4e45-b1d4-a09aa7dae672 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:33.807308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ab093-943e-4e45-b1d4-a09aa7dae672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:33.842477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1f2ab093-943e-4e45-b1d4-a09aa7dae672 None None] MFA Rules not processed for user `60367bf464f4441483bfe4a1373a1e92`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:33.842477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1f2ab093-943e-4e45-b1d4-a09aa7dae672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:34.016226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1126/2264] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:34.018953 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f105b8-7e80-44cb-8eb0-b272d12ab3c7 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:34.020808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-22f105b8-7e80-44cb-8eb0-b272d12ab3c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:34.138970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f105b8-7e80-44cb-8eb0-b272d12ab3c7 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:34.149716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bae8a-890e-4a6d-b61d-f501157e5d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:34.150562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bae8a-890e-4a6d-b61d-f501157e5d6a service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:34.151014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bae8a-890e-4a6d-b61d-f501157e5d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:34.151382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bae8a-890e-4a6d-b61d-f501157e5d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:34.175764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bae8a-890e-4a6d-b61d-f501157e5d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:34.178932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bae8a-890e-4a6d-b61d-f501157e5d6a service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:34.207986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1139/2265] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:35.924778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e251604-0137-4fdd-bff7-22d68492f324 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:35.924778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e251604-0137-4fdd-bff7-22d68492f324 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:35.924778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e251604-0137-4fdd-bff7-22d68492f324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:35.945723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1e251604-0137-4fdd-bff7-22d68492f324 None None] MFA Rules not processed for user `60367bf464f4441483bfe4a1373a1e92`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:35.959476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1e251604-0137-4fdd-bff7-22d68492f324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:35.967060 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1127/2266] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:39.279906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bae8a-890e-4a6d-b61d-f501157e5d6a service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:39.282035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-889bae8a-890e-4a6d-b61d-f501157e5d6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:39.310601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bae8a-890e-4a6d-b61d-f501157e5d6a service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:39.320412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:39.321983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:39.322391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:39.322753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] PATH_INFO: `/v3/users/9c12d080b671442580f33071a0a8efab` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:39.330387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c12d080b671442580f33071a0a8efab)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:39.334156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:39.414723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c12d080b671442580f33071a0a8efab'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:39.429802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c12d080b671442580f33071a0a8efab'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:39.444651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c12d080b671442580f33071a0a8efab'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:39.445672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c12d080b671442580f33071a0a8efab'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:39.470315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c12d080b671442580f33071a0a8efab'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:39.496794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1140/2267] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:39 2022] DELETE /identity/v3/users/9c12d080b671442580f33071a0a8efab => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:39.777430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e251604-0137-4fdd-bff7-22d68492f324 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:39.826109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e251604-0137-4fdd-bff7-22d68492f324 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:39.919196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:39.922690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:39.923409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:39.923904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] PATH_INFO: `/v3/projects/64524e73ca0f405cbccd891aa3564988` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:39.925976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64524e73ca0f405cbccd891aa3564988)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:39.928527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:39.997362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64524e73ca0f405cbccd891aa3564988', 'request_id': 'req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.004573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64524e73ca0f405cbccd891aa3564988', 'request_id': 'req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.008484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:40.019062 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64524e73ca0f405cbccd891aa3564988', 'request_id': 'req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.039389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] The token cache is being invalidate because project 64524e73ca0f405cbccd891aa3564988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:40.040004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.042568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.045763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1128/2268] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:39 2022] DELETE /identity/v3/projects/64524e73ca0f405cbccd891aa3564988 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:40.050146 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:40.051870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:40.070639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19eb34-3ff2-42f8-a94d-a2bb261d3cbf admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:40.086890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:40.262981 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:40.264138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:40.264624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:40.265169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] PATH_INFO: `/v3/users/c3fc03513df6437ba52b3fbee7175e87` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:40.267054 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3fc03513df6437ba52b3fbee7175e87)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:40.270128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:40.405645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3fc03513df6437ba52b3fbee7175e87'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.430757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3fc03513df6437ba52b3fbee7175e87'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.462219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3fc03513df6437ba52b3fbee7175e87'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.463410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3fc03513df6437ba52b3fbee7175e87'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.485061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-83f45c38-a756-43fd-986e-532e1f5f3f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3fc03513df6437ba52b3fbee7175e87'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:40.500130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1141/2269] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:40 2022] DELETE /identity/v3/users/c3fc03513df6437ba52b3fbee7175e87 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:41.135035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:41.137756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:41.163149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba12ae-84ce-48fc-bcfc-7312c6a0f642 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:41.261548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:41.263239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:41.263879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:41.264496 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] PATH_INFO: `/v3/projects/e66fea7a5cd440a58485e2884225d236` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:41.267139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e66fea7a5cd440a58485e2884225d236)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:41.270033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:41.376364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e66fea7a5cd440a58485e2884225d236', 'request_id': 'req-1b14f027-6e04-4d44-81ff-9777d2c04caf'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:41.394159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e66fea7a5cd440a58485e2884225d236', 'request_id': 'req-1b14f027-6e04-4d44-81ff-9777d2c04caf'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:41.413112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e66fea7a5cd440a58485e2884225d236', 'request_id': 'req-1b14f027-6e04-4d44-81ff-9777d2c04caf'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:41.445399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] The token cache is being invalidate because project e66fea7a5cd440a58485e2884225d236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:41.445399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:41.445399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b14f027-6e04-4d44-81ff-9777d2c04caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:41.445399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1129/2270] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:41 2022] DELETE /identity/v3/projects/e66fea7a5cd440a58485e2884225d236 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:41.451370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef0781-9660-4a19-bff3-f566671cb797 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:41.452489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef0781-9660-4a19-bff3-f566671cb797 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:41.461682 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef0781-9660-4a19-bff3-f566671cb797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:41.492867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f4ef0781-9660-4a19-bff3-f566671cb797 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:41.494493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f4ef0781-9660-4a19-bff3-f566671cb797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:41.609252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1142/2271] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:41.612152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2da58-6770-4692-bae7-30a9940c4e70 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:41.612447 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2da58-6770-4692-bae7-30a9940c4e70 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:41.612729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2da58-6770-4692-bae7-30a9940c4e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:41.650008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a4d2da58-6770-4692-bae7-30a9940c4e70 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:41.650386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a4d2da58-6770-4692-bae7-30a9940c4e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:41.669560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1130/2272] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:41.674692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef0781-9660-4a19-bff3-f566671cb797 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:41.688633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef0781-9660-4a19-bff3-f566671cb797 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:41.703012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd186589-2005-4145-b136-3fa705d78c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:41.707648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd186589-2005-4145-b136-3fa705d78c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:41.711792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd186589-2005-4145-b136-3fa705d78c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:41.712461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd186589-2005-4145-b136-3fa705d78c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:41.714014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd186589-2005-4145-b136-3fa705d78c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:37:41.714603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd186589-2005-4145-b136-3fa705d78c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:41.718624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd186589-2005-4145-b136-3fa705d78c06 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:41.736808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1143/2273] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:37:41.741124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2da58-6770-4692-bae7-30a9940c4e70 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:41.767218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2da58-6770-4692-bae7-30a9940c4e70 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:41.775209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953dc38-07b8-4c73-bcea-ecb83bef16ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:41.776094 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953dc38-07b8-4c73-bcea-ecb83bef16ca admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:41.776549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953dc38-07b8-4c73-bcea-ecb83bef16ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:41.778288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953dc38-07b8-4c73-bcea-ecb83bef16ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:41.779603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953dc38-07b8-4c73-bcea-ecb83bef16ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:41.782425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953dc38-07b8-4c73-bcea-ecb83bef16ca admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:41.837479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1131/2274] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:41 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:41.846545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd186589-2005-4145-b136-3fa705d78c06 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:41.880608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd186589-2005-4145-b136-3fa705d78c06 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:41.915787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:42.040074 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357659c5-d451-419f-8249-239e3aac697c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:42.040638 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357659c5-d451-419f-8249-239e3aac697c admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:42.041050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357659c5-d451-419f-8249-239e3aac697c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:42.042167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357659c5-d451-419f-8249-239e3aac697c admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:42.042928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357659c5-d451-419f-8249-239e3aac697c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:42.046489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357659c5-d451-419f-8249-239e3aac697c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:42.081804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1144/2275] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:41 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:42.086457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953dc38-07b8-4c73-bcea-ecb83bef16ca admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:42.109963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953dc38-07b8-4c73-bcea-ecb83bef16ca admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:42.118715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1887c-b773-43fc-bd0c-701beac36d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:42.119428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1887c-b773-43fc-bd0c-701beac36d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:42.119782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1887c-b773-43fc-bd0c-701beac36d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:42.120004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1887c-b773-43fc-bd0c-701beac36d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:42.120772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1887c-b773-43fc-bd0c-701beac36d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:42.123097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1887c-b773-43fc-bd0c-701beac36d5f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:42.149282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1132/2276] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:42.153104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357659c5-d451-419f-8249-239e3aac697c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:42.168617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357659c5-d451-419f-8249-239e3aac697c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:42.183412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9731d-3196-4013-a898-74168f2a8d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:42.184139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9731d-3196-4013-a898-74168f2a8d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:42.184427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9731d-3196-4013-a898-74168f2a8d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:42.184703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9731d-3196-4013-a898-74168f2a8d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:42.185472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9731d-3196-4013-a898-74168f2a8d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:42.187104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9731d-3196-4013-a898-74168f2a8d30 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:42.217922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1145/2277] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:42.221787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1887c-b773-43fc-bd0c-701beac36d5f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:42.235865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1887c-b773-43fc-bd0c-701beac36d5f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:42.241151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b00893-74bb-4a7e-aa35-01b60e32ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:42.242450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b00893-74bb-4a7e-aa35-01b60e32ce71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:42.242946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b00893-74bb-4a7e-aa35-01b60e32ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:42.243451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b00893-74bb-4a7e-aa35-01b60e32ce71 admin admin] PATH_INFO: `/v3/projects/4d8b3fc87bff4f1c8f2e4a4acb5df022/users/798ee2300cfb48a890a0d1a57f5a7786/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:42.268841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b00893-74bb-4a7e-aa35-01b60e32ce71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8b3fc87bff4f1c8f2e4a4acb5df022, user_id=798ee2300cfb48a890a0d1a57f5a7786, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:42.272918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b00893-74bb-4a7e-aa35-01b60e32ce71 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:42.307995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1133/2278] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:42 2022] PUT /identity/v3/projects/4d8b3fc87bff4f1c8f2e4a4acb5df022/users/798ee2300cfb48a890a0d1a57f5a7786/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:42.319042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7b245-3ddd-4cc8-9398-05d479ca9cc4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:42.319409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7b245-3ddd-4cc8-9398-05d479ca9cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:42.319662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7b245-3ddd-4cc8-9398-05d479ca9cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:42.378720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-93f7b245-3ddd-4cc8-9398-05d479ca9cc4 None None] MFA Rules not processed for user `798ee2300cfb48a890a0d1a57f5a7786`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:42.380649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-93f7b245-3ddd-4cc8-9398-05d479ca9cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:42.488163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1146/2279] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:42.496948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b00893-74bb-4a7e-aa35-01b60e32ce71 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:42.498883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-90b00893-74bb-4a7e-aa35-01b60e32ce71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:42.515618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:42.575043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b00893-74bb-4a7e-aa35-01b60e32ce71 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:42.581309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce906646-6ea5-42c0-89a8-ac15bc90e9a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:42.582499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce906646-6ea5-42c0-89a8-ac15bc90e9a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:42.583074 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce906646-6ea5-42c0-89a8-ac15bc90e9a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:42.583522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce906646-6ea5-42c0-89a8-ac15bc90e9a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:42.597591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce906646-6ea5-42c0-89a8-ac15bc90e9a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:42.601779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce906646-6ea5-42c0-89a8-ac15bc90e9a5 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:42.626699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1134/2280] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:42.692343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1971580-cd05-4323-813c-f368d050e31a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:42.692977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1971580-cd05-4323-813c-f368d050e31a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:42.693474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1971580-cd05-4323-813c-f368d050e31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:42.724823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c1971580-cd05-4323-813c-f368d050e31a None None] MFA Rules not processed for user `960b18fe180f4185b291da1d9c18a2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:42.726545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c1971580-cd05-4323-813c-f368d050e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:42.828723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1147/2281] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:45.521875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce906646-6ea5-42c0-89a8-ac15bc90e9a5 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:45.597565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce906646-6ea5-42c0-89a8-ac15bc90e9a5 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:45.602380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:45.603578 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:45.603970 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:45.604420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] PATH_INFO: `/v3/users/798ee2300cfb48a890a0d1a57f5a7786` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:45.606033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=798ee2300cfb48a890a0d1a57f5a7786)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:45.608139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:45.665117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '798ee2300cfb48a890a0d1a57f5a7786'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:45.681155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '798ee2300cfb48a890a0d1a57f5a7786'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:45.691635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '798ee2300cfb48a890a0d1a57f5a7786'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:45.699440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '798ee2300cfb48a890a0d1a57f5a7786'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:45.700201 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '798ee2300cfb48a890a0d1a57f5a7786'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:45.706815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1135/2282] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:45 2022] DELETE /identity/v3/users/798ee2300cfb48a890a0d1a57f5a7786 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:45.711688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d175c9-342a-4450-885d-e5615c971de1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:45.712230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d175c9-342a-4450-885d-e5615c971de1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:45.712326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d175c9-342a-4450-885d-e5615c971de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:45.732678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-83d175c9-342a-4450-885d-e5615c971de1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:45.734037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-83d175c9-342a-4450-885d-e5615c971de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:45.755516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:45.806512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1148/2283] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:45.817078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:45.818769 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:45.856519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:45.881779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cd00a-4017-4408-a2ca-8f2eff2c37a5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:45.885580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c1e8-951f-4203-a72c-21a5e2f3e600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:45.886238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c1e8-951f-4203-a72c-21a5e2f3e600 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:45.886521 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c1e8-951f-4203-a72c-21a5e2f3e600 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:45.886851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c1e8-951f-4203-a72c-21a5e2f3e600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:45.897552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13c1e8-951f-4203-a72c-21a5e2f3e600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:45.898985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13c1e8-951f-4203-a72c-21a5e2f3e600 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:45.913740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1136/2284] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:46.009526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30498398-a79e-4a07-b225-0ef959060c09 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:46.009951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30498398-a79e-4a07-b225-0ef959060c09 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:46.010103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30498398-a79e-4a07-b225-0ef959060c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:46.033988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-30498398-a79e-4a07-b225-0ef959060c09 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:46.035216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-30498398-a79e-4a07-b225-0ef959060c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:46.101135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1149/2285] 10.222.0.69 () {58 vars in 967 bytes} [Fri Jun 24 10:37:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:46.116054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c1e8-951f-4203-a72c-21a5e2f3e600 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:46.117600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bb13c1e8-951f-4203-a72c-21a5e2f3e600 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:46.203328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c1e8-951f-4203-a72c-21a5e2f3e600 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:46.209458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac8228-6f56-4db8-b47a-dd9cf5a015df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:46.210138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac8228-6f56-4db8-b47a-dd9cf5a015df service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:46.210438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac8228-6f56-4db8-b47a-dd9cf5a015df service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:46.210698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac8228-6f56-4db8-b47a-dd9cf5a015df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:46.220472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac8228-6f56-4db8-b47a-dd9cf5a015df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:46.222144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac8228-6f56-4db8-b47a-dd9cf5a015df service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:46.234903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1137/2286] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:37:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:37:46.265965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30498398-a79e-4a07-b225-0ef959060c09 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:46.267680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-30498398-a79e-4a07-b225-0ef959060c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:46.290899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30498398-a79e-4a07-b225-0ef959060c09 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:46.297896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:46.298994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:46.299214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:46.299787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] PATH_INFO: `/v3/projects/4d8b3fc87bff4f1c8f2e4a4acb5df022` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:46.301136 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d8b3fc87bff4f1c8f2e4a4acb5df022)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:46.303167 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:46.372661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d8b3fc87bff4f1c8f2e4a4acb5df022', 'request_id': 'req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:46.386323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d8b3fc87bff4f1c8f2e4a4acb5df022', 'request_id': 'req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:46.396951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d8b3fc87bff4f1c8f2e4a4acb5df022', 'request_id': 'req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:46.412549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] The token cache is being invalidate because project 4d8b3fc87bff4f1c8f2e4a4acb5df022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:46.413169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:46.414095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e50c1dd6-ace6-40cc-a9e4-ba5a4538f2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:46.416644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1150/2287] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:46 2022] DELETE /identity/v3/projects/4d8b3fc87bff4f1c8f2e4a4acb5df022 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:46.422569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d676-c634-4e3a-ae8d-4861687ab9c5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:46.423007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d676-c634-4e3a-ae8d-4861687ab9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:46.423491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d676-c634-4e3a-ae8d-4861687ab9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:46.460214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e408d676-c634-4e3a-ae8d-4861687ab9c5 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:46.461887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e408d676-c634-4e3a-ae8d-4861687ab9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:46.576383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1138/2288] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:37:46.583956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede20d9-05bc-4a62-9047-a4f042c84c54 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:46.584250 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede20d9-05bc-4a62-9047-a4f042c84c54 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:46.584533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede20d9-05bc-4a62-9047-a4f042c84c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:46.607417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bede20d9-05bc-4a62-9047-a4f042c84c54 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:46.608768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bede20d9-05bc-4a62-9047-a4f042c84c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:46.621091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1151/2289] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:46.622815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d676-c634-4e3a-ae8d-4861687ab9c5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:46.634239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d676-c634-4e3a-ae8d-4861687ab9c5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:46.640356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f037225-1a60-49a5-93a4-a4ddad633fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:46.641078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f037225-1a60-49a5-93a4-a4ddad633fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:46.641353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f037225-1a60-49a5-93a4-a4ddad633fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:46.641633 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f037225-1a60-49a5-93a4-a4ddad633fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:46.642382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f037225-1a60-49a5-93a4-a4ddad633fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:37:46.642693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f037225-1a60-49a5-93a4-a4ddad633fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:46.644309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f037225-1a60-49a5-93a4-a4ddad633fd9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:46.663231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1139/2290] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:37:46.673199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede20d9-05bc-4a62-9047-a4f042c84c54 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:46.676608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede20d9-05bc-4a62-9047-a4f042c84c54 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:46.682232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839d871-c9b6-40ae-aa87-b0099d07536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:46.683059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d871-c9b6-40ae-aa87-b0099d07536d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:46.683338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d871-c9b6-40ae-aa87-b0099d07536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:46.683819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d871-c9b6-40ae-aa87-b0099d07536d admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:46.684653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839d871-c9b6-40ae-aa87-b0099d07536d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:46.686823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839d871-c9b6-40ae-aa87-b0099d07536d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:46.736693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1152/2291] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:46 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:46.740622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f037225-1a60-49a5-93a4-a4ddad633fd9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:46.752796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f037225-1a60-49a5-93a4-a4ddad633fd9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:46.837566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:46.882404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fd7eb-a39b-4661-ae95-eb2317d4a3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:46.883123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fd7eb-a39b-4661-ae95-eb2317d4a3bb admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:46.883448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fd7eb-a39b-4661-ae95-eb2317d4a3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:46.883676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fd7eb-a39b-4661-ae95-eb2317d4a3bb admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:46.884374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fd7eb-a39b-4661-ae95-eb2317d4a3bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:46.886337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fd7eb-a39b-4661-ae95-eb2317d4a3bb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:46.938209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1140/2292] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:46 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:46.943194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839d871-c9b6-40ae-aa87-b0099d07536d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:46.972699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839d871-c9b6-40ae-aa87-b0099d07536d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:46.981341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cca8c-100e-4e4e-8527-55f8790e0866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:46.982205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cca8c-100e-4e4e-8527-55f8790e0866 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:46.982527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cca8c-100e-4e4e-8527-55f8790e0866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:46.982909 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cca8c-100e-4e4e-8527-55f8790e0866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:46.983661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1cca8c-100e-4e4e-8527-55f8790e0866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:46.987192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1cca8c-100e-4e4e-8527-55f8790e0866 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:47.008783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1153/2293] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:47.011525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fd7eb-a39b-4661-ae95-eb2317d4a3bb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:47.024716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fd7eb-a39b-4661-ae95-eb2317d4a3bb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:47.039167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b29a6-398f-41f4-9896-9aef95429b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:47.039942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b29a6-398f-41f4-9896-9aef95429b7c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:47.040217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b29a6-398f-41f4-9896-9aef95429b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:47.040510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b29a6-398f-41f4-9896-9aef95429b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:47.041368 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b29a6-398f-41f4-9896-9aef95429b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:47.043173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b29a6-398f-41f4-9896-9aef95429b7c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:47.066632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cca8c-100e-4e4e-8527-55f8790e0866 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:47.068026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1141/2294] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:47.078770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cca8c-100e-4e4e-8527-55f8790e0866 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:47.087538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d1002-d2a6-414b-b6a8-a185592e0a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:47.088579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d1002-d2a6-414b-b6a8-a185592e0a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:47.089051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d1002-d2a6-414b-b6a8-a185592e0a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:47.089314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d1002-d2a6-414b-b6a8-a185592e0a11 admin admin] PATH_INFO: `/v3/projects/96656242fbcb4727bb748e526496885e/users/d7acc8836a1e4863997090c8f58607b3/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:47.104692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:47.123686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d1002-d2a6-414b-b6a8-a185592e0a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96656242fbcb4727bb748e526496885e, user_id=d7acc8836a1e4863997090c8f58607b3, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:47.129735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d1002-d2a6-414b-b6a8-a185592e0a11 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:47.155830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1154/2295] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:47 2022] PUT /identity/v3/projects/96656242fbcb4727bb748e526496885e/users/d7acc8836a1e4863997090c8f58607b3/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:47.169229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe517cac-e03f-4f6c-8428-3acae65896ba None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:47.170206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe517cac-e03f-4f6c-8428-3acae65896ba None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:47.170511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe517cac-e03f-4f6c-8428-3acae65896ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:47.213075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fe517cac-e03f-4f6c-8428-3acae65896ba None None] MFA Rules not processed for user `d7acc8836a1e4863997090c8f58607b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:47.214762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fe517cac-e03f-4f6c-8428-3acae65896ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:47.289895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1142/2296] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:47.301733 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d1002-d2a6-414b-b6a8-a185592e0a11 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:47.303630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b34d1002-d2a6-414b-b6a8-a185592e0a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:47.373915 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d1002-d2a6-414b-b6a8-a185592e0a11 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:47.377644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9e434-845c-41bc-a4ff-33d5593d0687 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b75a27126c9464989cec70e5104824c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:47.378401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9e434-845c-41bc-a4ff-33d5593d0687 service glance] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:47.378743 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9e434-845c-41bc-a4ff-33d5593d0687 service glance] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:47.379080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9e434-845c-41bc-a4ff-33d5593d0687 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:47.388106 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9e434-845c-41bc-a4ff-33d5593d0687 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:47.389687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9e434-845c-41bc-a4ff-33d5593d0687 service glance] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:47.399463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1155/2297] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:37:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:37:48.775550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe517cac-e03f-4f6c-8428-3acae65896ba None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:48.777793 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fe517cac-e03f-4f6c-8428-3acae65896ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:48.903553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe517cac-e03f-4f6c-8428-3acae65896ba None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:48.908446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:48.909447 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:48.909731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:48.910009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] PATH_INFO: `/v3/users/57e63c7198d2450698858f9b84393c43` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:48.911885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e63c7198d2450698858f9b84393c43)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:48.913981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:48.993512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e63c7198d2450698858f9b84393c43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.033949 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:49.036985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e63c7198d2450698858f9b84393c43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.054905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e63c7198d2450698858f9b84393c43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.068281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e63c7198d2450698858f9b84393c43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.069136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-011a5372-0729-4746-ba4a-1778a9b1a9a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e63c7198d2450698858f9b84393c43'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.078671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1143/2298] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:37:48 2022] DELETE /identity/v3/users/57e63c7198d2450698858f9b84393c43 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:49.335701 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9e434-845c-41bc-a4ff-33d5593d0687 service glance] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:49.350855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9e434-845c-41bc-a4ff-33d5593d0687 service glance] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:49.424487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:49.425388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:49.425686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:49.425947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] PATH_INFO: `/v3/projects/9513ee5af8e84d48a7f28b4c77db2dd0` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:49.427298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9513ee5af8e84d48a7f28b4c77db2dd0)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:49.430379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:49.499685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9513ee5af8e84d48a7f28b4c77db2dd0', 'request_id': 'req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.509294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9513ee5af8e84d48a7f28b4c77db2dd0', 'request_id': 'req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.515794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9513ee5af8e84d48a7f28b4c77db2dd0', 'request_id': 'req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.527259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] The token cache is being invalidate because project 9513ee5af8e84d48a7f28b4c77db2dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:37:49.527543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.528177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e3f3cfac-0330-4f1a-9cb8-c51f10c917cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:37:49.529742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1156/2299] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:37:49 2022] DELETE /identity/v3/projects/9513ee5af8e84d48a7f28b4c77db2dd0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:37:49.534987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282bf1b-5052-4702-90c0-f32c790ae7c9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:49.535277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282bf1b-5052-4702-90c0-f32c790ae7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:49.535572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282bf1b-5052-4702-90c0-f32c790ae7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:49.554468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1282bf1b-5052-4702-90c0-f32c790ae7c9 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:49.555692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1282bf1b-5052-4702-90c0-f32c790ae7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:49.624490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1144/2300] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:37:49.634188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f27266-a0c7-403a-b6c5-bca56b630b5a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:49.634485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f27266-a0c7-403a-b6c5-bca56b630b5a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:49.634764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f27266-a0c7-403a-b6c5-bca56b630b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:49.656271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a9f27266-a0c7-403a-b6c5-bca56b630b5a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:49.657544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a9f27266-a0c7-403a-b6c5-bca56b630b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:49.666453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1157/2301] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:37:49.671091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282bf1b-5052-4702-90c0-f32c790ae7c9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:49.681409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282bf1b-5052-4702-90c0-f32c790ae7c9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:49.686098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da24d4a-3d12-43aa-ac5a-bbb90f60e7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:49.686770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da24d4a-3d12-43aa-ac5a-bbb90f60e7fa admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:49.687057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da24d4a-3d12-43aa-ac5a-bbb90f60e7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:49.687336 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da24d4a-3d12-43aa-ac5a-bbb90f60e7fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:49.688072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da24d4a-3d12-43aa-ac5a-bbb90f60e7fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:37:49.688378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da24d4a-3d12-43aa-ac5a-bbb90f60e7fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:49.690187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da24d4a-3d12-43aa-ac5a-bbb90f60e7fa admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:49.706450 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1145/2302] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:37:49.707050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f27266-a0c7-403a-b6c5-bca56b630b5a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:49.717688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f27266-a0c7-403a-b6c5-bca56b630b5a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:49.724370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81724-b4bb-4ede-bf3b-f54a318f4f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:49.725170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81724-b4bb-4ede-bf3b-f54a318f4f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:49.725521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81724-b4bb-4ede-bf3b-f54a318f4f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:49.725893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81724-b4bb-4ede-bf3b-f54a318f4f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:49.726806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81724-b4bb-4ede-bf3b-f54a318f4f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:49.728696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81724-b4bb-4ede-bf3b-f54a318f4f3e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:49.743643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:49.761170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1158/2303] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:37:49 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:37:49.765971 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da24d4a-3d12-43aa-ac5a-bbb90f60e7fa admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:49.776815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da24d4a-3d12-43aa-ac5a-bbb90f60e7fa admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:49.834300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108e34b-c8bd-4770-9558-20790169eba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:49.834969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108e34b-c8bd-4770-9558-20790169eba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:49.835250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108e34b-c8bd-4770-9558-20790169eba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:49.835518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108e34b-c8bd-4770-9558-20790169eba8 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:49.836197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108e34b-c8bd-4770-9558-20790169eba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:49.838129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108e34b-c8bd-4770-9558-20790169eba8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:49.860413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1146/2304] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:37:49 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:37:49.864599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81724-b4bb-4ede-bf3b-f54a318f4f3e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:49.875699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81724-b4bb-4ede-bf3b-f54a318f4f3e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:49.883012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90973dd1-0639-44b2-b887-3ac58290a9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:49.883995 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90973dd1-0639-44b2-b887-3ac58290a9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:49.884279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90973dd1-0639-44b2-b887-3ac58290a9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:49.884554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90973dd1-0639-44b2-b887-3ac58290a9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:49.885428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90973dd1-0639-44b2-b887-3ac58290a9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:49.887001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90973dd1-0639-44b2-b887-3ac58290a9aa admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:49.912783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1159/2305] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:49.918045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108e34b-c8bd-4770-9558-20790169eba8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:49.930139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108e34b-c8bd-4770-9558-20790169eba8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:49.936087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5de710-9a4f-4538-b9fa-a49e8af06ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:49.937018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de710-9a4f-4538-b9fa-a49e8af06ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:49.937132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de710-9a4f-4538-b9fa-a49e8af06ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:49.937371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de710-9a4f-4538-b9fa-a49e8af06ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:49.938112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5de710-9a4f-4538-b9fa-a49e8af06ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:49.939724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5de710-9a4f-4538-b9fa-a49e8af06ca9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:49.953850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1147/2306] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:37:49.955027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90973dd1-0639-44b2-b887-3ac58290a9aa admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:37:49.968556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90973dd1-0639-44b2-b887-3ac58290a9aa admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:37:49.974925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587f926-5d85-4398-91d3-01ac9c18199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:37:49.976358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587f926-5d85-4398-91d3-01ac9c18199b admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:49.976832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587f926-5d85-4398-91d3-01ac9c18199b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:49.977217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587f926-5d85-4398-91d3-01ac9c18199b admin admin] PATH_INFO: `/v3/projects/218301fa11434ca5a291440479975e76/users/ce496b2ad31e446f8dab37b008be6ece/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:49.994405 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587f926-5d85-4398-91d3-01ac9c18199b admin admin] RBAC: Authorizing `identity:create_grant(project_id=218301fa11434ca5a291440479975e76, user_id=ce496b2ad31e446f8dab37b008be6ece, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:37:49.998190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587f926-5d85-4398-91d3-01ac9c18199b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:37:50.015553 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1160/2307] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:37:49 2022] PUT /identity/v3/projects/218301fa11434ca5a291440479975e76/users/ce496b2ad31e446f8dab37b008be6ece/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:37:50.026215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3363688-2ddb-4456-8f84-ee55991bc983 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:37:50.026835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3363688-2ddb-4456-8f84-ee55991bc983 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:37:50.027118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3363688-2ddb-4456-8f84-ee55991bc983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:37:50.083711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b3363688-2ddb-4456-8f84-ee55991bc983 None None] MFA Rules not processed for user `ce496b2ad31e446f8dab37b008be6ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:37:50.084628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b3363688-2ddb-4456-8f84-ee55991bc983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:37:50.136915 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:37:50.177938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1148/2308] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:38:00.011331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=17,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:00.175107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=11 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:03.528764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587f926-5d85-4398-91d3-01ac9c18199b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:03.531444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3587f926-5d85-4398-91d3-01ac9c18199b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:03.627468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587f926-5d85-4398-91d3-01ac9c18199b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:03.631755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ebb5d-f5fd-4442-b529-bf5233b4fdac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b75a27126c9464989cec70e5104824c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:03.632410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ebb5d-f5fd-4442-b529-bf5233b4fdac service glance] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:03.632692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ebb5d-f5fd-4442-b529-bf5233b4fdac service glance] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:03.633001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ebb5d-f5fd-4442-b529-bf5233b4fdac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:03.654845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ebb5d-f5fd-4442-b529-bf5233b4fdac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:03.656341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ebb5d-f5fd-4442-b529-bf5233b4fdac service glance] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:03.668705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1161/2309] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:38:04.014575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6092f-2721-4cd1-9ac1-3e89f5bc7f8c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:04.015200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6092f-2721-4cd1-9ac1-3e89f5bc7f8c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:04.015770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6092f-2721-4cd1-9ac1-3e89f5bc7f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:04.035611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7ce6092f-2721-4cd1-9ac1-3e89f5bc7f8c None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:04.037104 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ce6092f-2721-4cd1-9ac1-3e89f5bc7f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:04.102210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1149/2310] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:38:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:04.115766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ebb5d-f5fd-4442-b529-bf5233b4fdac service glance] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:04.117579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f02ebb5d-f5fd-4442-b529-bf5233b4fdac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:04.214259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ebb5d-f5fd-4442-b529-bf5233b4fdac service glance] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:04.219715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ea250-d43f-41be-9116-050614845679 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:04.220377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ea250-d43f-41be-9116-050614845679 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:04.220661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ea250-d43f-41be-9116-050614845679 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:04.220963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ea250-d43f-41be-9116-050614845679 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:04.229429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ea250-d43f-41be-9116-050614845679 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:04.231036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ea250-d43f-41be-9116-050614845679 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:04.240474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1162/2311] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:38:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:38:14.095441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=25 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:14.237190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=80 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:18.344962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce12586-2a65-462e-99a0-87a72954b640 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:18.345565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce12586-2a65-462e-99a0-87a72954b640 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:18.345674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce12586-2a65-462e-99a0-87a72954b640 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:18.347116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1150/2312] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:38:18.353119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cb425-117b-4eff-ad89-ebf382168c38 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:18.353596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cb425-117b-4eff-ad89-ebf382168c38 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:18.353785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cb425-117b-4eff-ad89-ebf382168c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:18.397415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-236cb425-117b-4eff-ad89-ebf382168c38 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:18.398624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-236cb425-117b-4eff-ad89-ebf382168c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:18.478722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1163/2313] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:38:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:38:18.490678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce12586-2a65-462e-99a0-87a72954b640 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:18.501461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce12586-2a65-462e-99a0-87a72954b640 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:18.505263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc0b6d-75c9-40a9-b8a8-76c722b6b5d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b75a27126c9464989cec70e5104824c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:18.505896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc0b6d-75c9-40a9-b8a8-76c722b6b5d5 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:18.506174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc0b6d-75c9-40a9-b8a8-76c722b6b5d5 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:18.506444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc0b6d-75c9-40a9-b8a8-76c722b6b5d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:18.517208 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc0b6d-75c9-40a9-b8a8-76c722b6b5d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:18.518756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc0b6d-75c9-40a9-b8a8-76c722b6b5d5 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:18.529464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1151/2314] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:38:18.603259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacc984-189e-44b5-a64f-09c5ff4412a5 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:18.603652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacc984-189e-44b5-a64f-09c5ff4412a5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:18.604004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacc984-189e-44b5-a64f-09c5ff4412a5 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:18.605617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1164/2315] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:38:18.616417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587645f-bc0c-4593-8bc5-5c3318caee4e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:18.616705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587645f-bc0c-4593-8bc5-5c3318caee4e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:18.617012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587645f-bc0c-4593-8bc5-5c3318caee4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:18.647844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8587645f-bc0c-4593-8bc5-5c3318caee4e None None] MFA Rules not processed for user `a86de22b0e324c70aad792ec6b1a7e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:18.649159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8587645f-bc0c-4593-8bc5-5c3318caee4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:18.712880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1152/2316] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:38:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:18.725099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacc984-189e-44b5-a64f-09c5ff4412a5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:18.726863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aaacc984-189e-44b5-a64f-09c5ff4412a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:18.740213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacc984-189e-44b5-a64f-09c5ff4412a5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:18.744352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ed084-9f66-4506-8c09-830a0a267dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:18.745016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ed084-9f66-4506-8c09-830a0a267dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:18.745288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ed084-9f66-4506-8c09-830a0a267dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:18.745563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ed084-9f66-4506-8c09-830a0a267dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:18.753104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ed084-9f66-4506-8c09-830a0a267dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:18.754537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ed084-9f66-4506-8c09-830a0a267dd9 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:18.765658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1165/2317] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:38:18.851561 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8c1de-d350-4f56-9a85-a96764ad30e8 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:18.851849 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8c1de-d350-4f56-9a85-a96764ad30e8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:18.852148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8c1de-d350-4f56-9a85-a96764ad30e8 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:18.853566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1153/2318] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:38:18.858895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca58e5-0c62-4bcb-abb3-26f330959094 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:18.859213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca58e5-0c62-4bcb-abb3-26f330959094 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:18.859496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca58e5-0c62-4bcb-abb3-26f330959094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:18.878672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-31ca58e5-0c62-4bcb-abb3-26f330959094 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:18.879877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-31ca58e5-0c62-4bcb-abb3-26f330959094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:18.890518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1166/2319] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:38:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:18.900257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8c1de-d350-4f56-9a85-a96764ad30e8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:18.912374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8c1de-d350-4f56-9a85-a96764ad30e8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:18.916057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade515f5-1b5a-4ed7-9fc2-e1a760b635a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:18.916693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade515f5-1b5a-4ed7-9fc2-e1a760b635a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:18.917009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade515f5-1b5a-4ed7-9fc2-e1a760b635a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:18.917281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade515f5-1b5a-4ed7-9fc2-e1a760b635a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:18.927833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade515f5-1b5a-4ed7-9fc2-e1a760b635a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:18.929318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade515f5-1b5a-4ed7-9fc2-e1a760b635a5 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:18.940154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1154/2320] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:38:21.479806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e558c47-a7d0-4554-9239-fdcc41fee313 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:21.480295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e558c47-a7d0-4554-9239-fdcc41fee313 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:21.481633 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e558c47-a7d0-4554-9239-fdcc41fee313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:21.512993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2e558c47-a7d0-4554-9239-fdcc41fee313 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:21.514696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2e558c47-a7d0-4554-9239-fdcc41fee313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:21.523318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1167/2321] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:38:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:21.536438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade515f5-1b5a-4ed7-9fc2-e1a760b635a5 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:21.550001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade515f5-1b5a-4ed7-9fc2-e1a760b635a5 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:21.558483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df7577-4c09-47f1-a7f3-7d4257882042 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:21.559136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df7577-4c09-47f1-a7f3-7d4257882042 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:21.559419 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df7577-4c09-47f1-a7f3-7d4257882042 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:21.559695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df7577-4c09-47f1-a7f3-7d4257882042 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:21.569971 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df7577-4c09-47f1-a7f3-7d4257882042 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:21.571906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df7577-4c09-47f1-a7f3-7d4257882042 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:21.582763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1155/2322] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:38:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:38:31.502788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=51 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:31.578507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=50 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:33.451487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e558c47-a7d0-4554-9239-fdcc41fee313 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:33.452929 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2e558c47-a7d0-4554-9239-fdcc41fee313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:33.552866 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e558c47-a7d0-4554-9239-fdcc41fee313 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:33.558340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2eac96-31e7-4c31-bb2a-5ebc50360768 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:33.558989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eac96-31e7-4c31-bb2a-5ebc50360768 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:33.559272 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eac96-31e7-4c31-bb2a-5ebc50360768 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:33.559553 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eac96-31e7-4c31-bb2a-5ebc50360768 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:33.561483 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ba2eac96-31e7-4c31-bb2a-5ebc50360768 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:33.572800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2eac96-31e7-4c31-bb2a-5ebc50360768 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:33.574377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2eac96-31e7-4c31-bb2a-5ebc50360768 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:33.584202 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1168/2323] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:38:38.899577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df7577-4c09-47f1-a7f3-7d4257882042 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:38.901655 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-68df7577-4c09-47f1-a7f3-7d4257882042 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:39.001458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df7577-4c09-47f1-a7f3-7d4257882042 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:39.006374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:39.007257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:39.007537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:39.007796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] PATH_INFO: `/v3/users/d7acc8836a1e4863997090c8f58607b3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:39.009084 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7acc8836a1e4863997090c8f58607b3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:39.011095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:39.065985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7acc8836a1e4863997090c8f58607b3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.079014 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7acc8836a1e4863997090c8f58607b3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.089266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7acc8836a1e4863997090c8f58607b3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.096880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7acc8836a1e4863997090c8f58607b3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.097523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7acc8836a1e4863997090c8f58607b3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.103490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1156/2324] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:38:38 2022] DELETE /identity/v3/users/d7acc8836a1e4863997090c8f58607b3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:38:39.109184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca0ef9-376a-49a0-aa94-3f2c7c4b3570 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:39.109650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca0ef9-376a-49a0-aa94-3f2c7c4b3570 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:39.109776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca0ef9-376a-49a0-aa94-3f2c7c4b3570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:39.140102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a4ca0ef9-376a-49a0-aa94-3f2c7c4b3570 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:39.141339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a4ca0ef9-376a-49a0-aa94-3f2c7c4b3570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:39.203580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1169/2325] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:38:39.214571 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:39.216282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:39.277918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144210db-e4a1-4d65-93bb-12994b781332 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:39.282263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6d0c7-aa24-4664-a6af-22b954ff4e64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:39.283145 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6d0c7-aa24-4664-a6af-22b954ff4e64 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:39.283314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6d0c7-aa24-4664-a6af-22b954ff4e64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:39.283641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6d0c7-aa24-4664-a6af-22b954ff4e64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:39.290976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:39.296295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6d0c7-aa24-4664-a6af-22b954ff4e64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:39.298246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6d0c7-aa24-4664-a6af-22b954ff4e64 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:39.308817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1157/2326] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:38:39.585409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca0ef9-376a-49a0-aa94-3f2c7c4b3570 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:39.586935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a4ca0ef9-376a-49a0-aa94-3f2c7c4b3570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:39.600003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca0ef9-376a-49a0-aa94-3f2c7c4b3570 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:39.605149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:39.606022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:39.606302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:39.606579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] PATH_INFO: `/v3/projects/96656242fbcb4727bb748e526496885e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:39.607774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] RBAC: Authorizing `identity:delete_project(project_id=96656242fbcb4727bb748e526496885e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:39.609739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:39.660554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96656242fbcb4727bb748e526496885e', 'request_id': 'req-7ce850b2-379c-4d12-9cc7-293746e8302c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.669309 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96656242fbcb4727bb748e526496885e', 'request_id': 'req-7ce850b2-379c-4d12-9cc7-293746e8302c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.671420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:39.675566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96656242fbcb4727bb748e526496885e', 'request_id': 'req-7ce850b2-379c-4d12-9cc7-293746e8302c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.693651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] The token cache is being invalidate because project 96656242fbcb4727bb748e526496885e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:38:39.693958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.694800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7ce850b2-379c-4d12-9cc7-293746e8302c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:39.696579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1170/2327] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:38:39 2022] DELETE /identity/v3/projects/96656242fbcb4727bb748e526496885e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:38:39.701948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7399cb-8175-49d0-9980-14dd8eb02a20 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:39.702236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7399cb-8175-49d0-9980-14dd8eb02a20 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:39.702512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7399cb-8175-49d0-9980-14dd8eb02a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:39.740028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9b7399cb-8175-49d0-9980-14dd8eb02a20 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:39.741251 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9b7399cb-8175-49d0-9980-14dd8eb02a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:39.801760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1158/2328] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:39.809896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f9c6-1e68-4f94-8d30-7f8ab7543a93 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:39.810188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f9c6-1e68-4f94-8d30-7f8ab7543a93 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:39.810487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f9c6-1e68-4f94-8d30-7f8ab7543a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:39.830459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-02f9f9c6-1e68-4f94-8d30-7f8ab7543a93 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:39.831589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-02f9f9c6-1e68-4f94-8d30-7f8ab7543a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:39.841048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1171/2329] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:39.846185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7399cb-8175-49d0-9980-14dd8eb02a20 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:39.862049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7399cb-8175-49d0-9980-14dd8eb02a20 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:39.870257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af89621-0da9-4ea2-80ec-3fbc36fcefa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:39.870916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af89621-0da9-4ea2-80ec-3fbc36fcefa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:39.871199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af89621-0da9-4ea2-80ec-3fbc36fcefa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:39.871473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af89621-0da9-4ea2-80ec-3fbc36fcefa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:39.872206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af89621-0da9-4ea2-80ec-3fbc36fcefa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:38:39.872510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af89621-0da9-4ea2-80ec-3fbc36fcefa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:39.874149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af89621-0da9-4ea2-80ec-3fbc36fcefa3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:39.887068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1159/2330] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:38:39.891630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9f9c6-1e68-4f94-8d30-7f8ab7543a93 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:39.901691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9f9c6-1e68-4f94-8d30-7f8ab7543a93 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:39.907737 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e0efd-a660-4eb7-96e6-66bf2a015a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:39.908401 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e0efd-a660-4eb7-96e6-66bf2a015a68 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:39.908683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e0efd-a660-4eb7-96e6-66bf2a015a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:39.908991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e0efd-a660-4eb7-96e6-66bf2a015a68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:39.909700 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e0efd-a660-4eb7-96e6-66bf2a015a68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:39.912049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e0efd-a660-4eb7-96e6-66bf2a015a68 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:39.947677 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1172/2331] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:38:39 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:38:39.948985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af89621-0da9-4ea2-80ec-3fbc36fcefa3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:39.959389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af89621-0da9-4ea2-80ec-3fbc36fcefa3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:40.019471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fddb5d-3100-46c4-91f4-9e94272815c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:40.020196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fddb5d-3100-46c4-91f4-9e94272815c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:40.020484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fddb5d-3100-46c4-91f4-9e94272815c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:40.021314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fddb5d-3100-46c4-91f4-9e94272815c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:40.021526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fddb5d-3100-46c4-91f4-9e94272815c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:40.023541 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fddb5d-3100-46c4-91f4-9e94272815c1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:40.049709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1160/2332] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:38:39 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:38:40.050983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e0efd-a660-4eb7-96e6-66bf2a015a68 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:40.065999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e0efd-a660-4eb7-96e6-66bf2a015a68 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:40.076244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788b9e-f896-44be-9635-9b06f50ebadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:40.076974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788b9e-f896-44be-9635-9b06f50ebadd admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:40.077279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788b9e-f896-44be-9635-9b06f50ebadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:40.077554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788b9e-f896-44be-9635-9b06f50ebadd admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:40.078300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b788b9e-f896-44be-9635-9b06f50ebadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:40.079906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b788b9e-f896-44be-9635-9b06f50ebadd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:40.091115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1173/2333] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:38:40.095428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fddb5d-3100-46c4-91f4-9e94272815c1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:40.105610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fddb5d-3100-46c4-91f4-9e94272815c1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:40.110118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc991c78-d7d9-48c7-ab6b-7c75bd063860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:40.110771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc991c78-d7d9-48c7-ab6b-7c75bd063860 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:40.111053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc991c78-d7d9-48c7-ab6b-7c75bd063860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:40.111326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc991c78-d7d9-48c7-ab6b-7c75bd063860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:40.112055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc991c78-d7d9-48c7-ab6b-7c75bd063860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:40.113691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc991c78-d7d9-48c7-ab6b-7c75bd063860 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:40.123583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1161/2334] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:38:40.127718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788b9e-f896-44be-9635-9b06f50ebadd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:40.138902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788b9e-f896-44be-9635-9b06f50ebadd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:40.144632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa93d3-43a8-4649-a5b0-899d1a848406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:40.145548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa93d3-43a8-4649-a5b0-899d1a848406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:40.145833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa93d3-43a8-4649-a5b0-899d1a848406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:40.146116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa93d3-43a8-4649-a5b0-899d1a848406 admin admin] PATH_INFO: `/v3/projects/8eb1e5f40033474c947399c22bb0a2d2/users/5fc073d480884782b4084952345382b9/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:40.169061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa93d3-43a8-4649-a5b0-899d1a848406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb1e5f40033474c947399c22bb0a2d2, user_id=5fc073d480884782b4084952345382b9, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:40.171727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa93d3-43a8-4649-a5b0-899d1a848406 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:40.188402 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1174/2335] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:38:40 2022] PUT /identity/v3/projects/8eb1e5f40033474c947399c22bb0a2d2/users/5fc073d480884782b4084952345382b9/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:38:40.193687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f019643-c001-44c7-bd6d-a7a058e7c3a4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:40.193998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f019643-c001-44c7-bd6d-a7a058e7c3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:40.194282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f019643-c001-44c7-bd6d-a7a058e7c3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:40.213742 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3f019643-c001-44c7-bd6d-a7a058e7c3a4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:40.214964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3f019643-c001-44c7-bd6d-a7a058e7c3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:40.266384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:40.287928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1162/2336] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:40.297884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa93d3-43a8-4649-a5b0-899d1a848406 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:40.299941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-52aa93d3-43a8-4649-a5b0-899d1a848406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:40.359415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa93d3-43a8-4649-a5b0-899d1a848406 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:40.363479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b5a81-be5c-491a-9cb2-775bc1ffd1d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:40.364127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b5a81-be5c-491a-9cb2-775bc1ffd1d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:40.364410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b5a81-be5c-491a-9cb2-775bc1ffd1d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:40.364681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b5a81-be5c-491a-9cb2-775bc1ffd1d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:40.374854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b5a81-be5c-491a-9cb2-775bc1ffd1d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:40.376280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b5a81-be5c-491a-9cb2-775bc1ffd1d4 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:40.386230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1175/2337] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:38:44.440886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433bfee-cd48-4d27-aa16-beb98ceb2ac2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:44.441717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433bfee-cd48-4d27-aa16-beb98ceb2ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:44.442165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433bfee-cd48-4d27-aa16-beb98ceb2ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:44.478149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a433bfee-cd48-4d27-aa16-beb98ceb2ac2 None None] MFA Rules not processed for user `5fc073d480884782b4084952345382b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:44.479398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a433bfee-cd48-4d27-aa16-beb98ceb2ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:44.568217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1163/2338] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:44.572441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b5a81-be5c-491a-9cb2-775bc1ffd1d4 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:44.584008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b5a81-be5c-491a-9cb2-775bc1ffd1d4 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:44.588963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572e5e8-3075-4f27-83fb-c73f9191c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:44.589645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572e5e8-3075-4f27-83fb-c73f9191c3bc admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:44.589928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572e5e8-3075-4f27-83fb-c73f9191c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:44.590207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572e5e8-3075-4f27-83fb-c73f9191c3bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:44.590922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572e5e8-3075-4f27-83fb-c73f9191c3bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:44.593413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572e5e8-3075-4f27-83fb-c73f9191c3bc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:44.623624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1176/2339] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:38:44 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:38:44.628008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433bfee-cd48-4d27-aa16-beb98ceb2ac2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:44.641174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433bfee-cd48-4d27-aa16-beb98ceb2ac2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:44.704955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8d876-84dd-4e2f-aab0-877964cab455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:44.705869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8d876-84dd-4e2f-aab0-877964cab455 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:44.706217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8d876-84dd-4e2f-aab0-877964cab455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:44.706552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8d876-84dd-4e2f-aab0-877964cab455 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:44.707318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8d876-84dd-4e2f-aab0-877964cab455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:44.710670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8d876-84dd-4e2f-aab0-877964cab455 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:44.734602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1164/2340] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:38:44 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:38:44.741042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572e5e8-3075-4f27-83fb-c73f9191c3bc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:44.759495 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572e5e8-3075-4f27-83fb-c73f9191c3bc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:44.765096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfaa49-0b83-47b5-aaf1-aecd0e2c430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:44.765845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfaa49-0b83-47b5-aaf1-aecd0e2c430a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:44.766268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfaa49-0b83-47b5-aaf1-aecd0e2c430a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:44.766750 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfaa49-0b83-47b5-aaf1-aecd0e2c430a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:44.767349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfaa49-0b83-47b5-aaf1-aecd0e2c430a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:44.769280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfaa49-0b83-47b5-aaf1-aecd0e2c430a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:44.785873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1177/2341] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:38:44.789815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8d876-84dd-4e2f-aab0-877964cab455 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:44.805632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8d876-84dd-4e2f-aab0-877964cab455 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:44.813786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b931f7-3f53-43b0-98f0-a967c9ff3638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:44.814755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b931f7-3f53-43b0-98f0-a967c9ff3638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:44.815040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b931f7-3f53-43b0-98f0-a967c9ff3638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:44.815313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b931f7-3f53-43b0-98f0-a967c9ff3638 admin admin] PATH_INFO: `/v3/projects/de5e5149971141d8ab26dbea50f0cf4e/users/c77c4c5e840444929f648cfb735e995f/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:44.832537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b931f7-3f53-43b0-98f0-a967c9ff3638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5e5149971141d8ab26dbea50f0cf4e, user_id=c77c4c5e840444929f648cfb735e995f, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:44.835342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b931f7-3f53-43b0-98f0-a967c9ff3638 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:44.852461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1165/2342] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:38:44 2022] PUT /identity/v3/projects/de5e5149971141d8ab26dbea50f0cf4e/users/c77c4c5e840444929f648cfb735e995f/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:38:44.856466 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfaa49-0b83-47b5-aaf1-aecd0e2c430a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:44.867460 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfaa49-0b83-47b5-aaf1-aecd0e2c430a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:44.932316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:44.942584 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebebfd4-3945-4bb4-bec3-ad40ff6877fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:44.943356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebebfd4-3945-4bb4-bec3-ad40ff6877fa admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:44.943893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebebfd4-3945-4bb4-bec3-ad40ff6877fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:44.944673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebebfd4-3945-4bb4-bec3-ad40ff6877fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:44.945690 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebebfd4-3945-4bb4-bec3-ad40ff6877fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:44.947601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebebfd4-3945-4bb4-bec3-ad40ff6877fa admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:44.964010 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1178/2343] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:38:44.976376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b931f7-3f53-43b0-98f0-a967c9ff3638 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:44.991483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b931f7-3f53-43b0-98f0-a967c9ff3638 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:44.996994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d513a0-1ba9-42d9-ab3e-d97374be8c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:44.997908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d513a0-1ba9-42d9-ab3e-d97374be8c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:44.998187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d513a0-1ba9-42d9-ab3e-d97374be8c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:44.998459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d513a0-1ba9-42d9-ab3e-d97374be8c37 admin admin] PATH_INFO: `/v3/projects/de5e5149971141d8ab26dbea50f0cf4e/users/c77c4c5e840444929f648cfb735e995f/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:45.000530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d513a0-1ba9-42d9-ab3e-d97374be8c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5e5149971141d8ab26dbea50f0cf4e, user_id=c77c4c5e840444929f648cfb735e995f, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:45.003385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d513a0-1ba9-42d9-ab3e-d97374be8c37 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:45.014522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1166/2344] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:38:44 2022] PUT /identity/v3/projects/de5e5149971141d8ab26dbea50f0cf4e/users/c77c4c5e840444929f648cfb735e995f/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:38:47.006649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebebfd4-3945-4bb4-bec3-ad40ff6877fa admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:47.011624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2ebebfd4-3945-4bb4-bec3-ad40ff6877fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:47.027400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebebfd4-3945-4bb4-bec3-ad40ff6877fa admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:47.115344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:47.116708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:47.121208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:47.121588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] PATH_INFO: `/v3/users/ce496b2ad31e446f8dab37b008be6ece` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:47.124817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce496b2ad31e446f8dab37b008be6ece)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:47.128234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:47.199983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce496b2ad31e446f8dab37b008be6ece'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:47.225937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce496b2ad31e446f8dab37b008be6ece'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:47.242336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce496b2ad31e446f8dab37b008be6ece'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:47.244416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce496b2ad31e446f8dab37b008be6ece'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:47.254831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce496b2ad31e446f8dab37b008be6ece'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:47.268724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1179/2345] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:38:47 2022] DELETE /identity/v3/users/ce496b2ad31e446f8dab37b008be6ece => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:38:47.272487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ade9ec-b3e1-44ed-b802-1cafe1225c6e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:47.273103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ade9ec-b3e1-44ed-b802-1cafe1225c6e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:47.273103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ade9ec-b3e1-44ed-b802-1cafe1225c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:47.293326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-58ade9ec-b3e1-44ed-b802-1cafe1225c6e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:47.295766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-58ade9ec-b3e1-44ed-b802-1cafe1225c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:47.320911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:47.403192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1167/2346] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:47.409852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:47.412360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:47.530499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acc402-cbdf-47bd-956b-107c799ca589 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:47.536669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb4ef9-d570-4ea2-8682-72664ea5b2c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:47.537611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb4ef9-d570-4ea2-8682-72664ea5b2c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:47.538034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb4ef9-d570-4ea2-8682-72664ea5b2c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:47.538446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb4ef9-d570-4ea2-8682-72664ea5b2c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:47.555961 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb4ef9-d570-4ea2-8682-72664ea5b2c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:47.560122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb4ef9-d570-4ea2-8682-72664ea5b2c4 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:47.579297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1180/2347] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:38:48.556120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ade9ec-b3e1-44ed-b802-1cafe1225c6e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:48.557766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-58ade9ec-b3e1-44ed-b802-1cafe1225c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:48.571508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ade9ec-b3e1-44ed-b802-1cafe1225c6e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:48.577122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:48.580206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:48.580593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:48.580994 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] PATH_INFO: `/v3/projects/218301fa11434ca5a291440479975e76` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:48.582526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] RBAC: Authorizing `identity:delete_project(project_id=218301fa11434ca5a291440479975e76)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:48.585540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:48.687896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '218301fa11434ca5a291440479975e76', 'request_id': 'req-0e027e92-4fea-4ea8-8588-66fc8b3b625d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:48.695774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '218301fa11434ca5a291440479975e76', 'request_id': 'req-0e027e92-4fea-4ea8-8588-66fc8b3b625d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:48.709854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '218301fa11434ca5a291440479975e76', 'request_id': 'req-0e027e92-4fea-4ea8-8588-66fc8b3b625d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:48.724505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] The token cache is being invalidate because project 218301fa11434ca5a291440479975e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:38:48.725001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:48.725850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e027e92-4fea-4ea8-8588-66fc8b3b625d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:38:48.728681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1168/2348] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:38:48 2022] DELETE /identity/v3/projects/218301fa11434ca5a291440479975e76 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:38:48.734157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b536a46-2bea-4a12-88ca-81e2899a620c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:48.734664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b536a46-2bea-4a12-88ca-81e2899a620c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:48.734767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b536a46-2bea-4a12-88ca-81e2899a620c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:48.762230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4b536a46-2bea-4a12-88ca-81e2899a620c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:48.763459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4b536a46-2bea-4a12-88ca-81e2899a620c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:48.770900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:48.862094 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1181/2349] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:48.870054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaca4a9-4fd1-46fa-aa9b-38bcb4cf9cbb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:48.870343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaca4a9-4fd1-46fa-aa9b-38bcb4cf9cbb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:48.870619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaca4a9-4fd1-46fa-aa9b-38bcb4cf9cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:48.892845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cbaca4a9-4fd1-46fa-aa9b-38bcb4cf9cbb None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:48.894579 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cbaca4a9-4fd1-46fa-aa9b-38bcb4cf9cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:48.909468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1169/2350] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:38:48.911552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b536a46-2bea-4a12-88ca-81e2899a620c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:48.922220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b536a46-2bea-4a12-88ca-81e2899a620c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:48.928300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf75ca0-5fff-467f-9f96-b7f7ffa5f374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:48.929174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf75ca0-5fff-467f-9f96-b7f7ffa5f374 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:48.929566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf75ca0-5fff-467f-9f96-b7f7ffa5f374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:48.929843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf75ca0-5fff-467f-9f96-b7f7ffa5f374 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:48.930581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf75ca0-5fff-467f-9f96-b7f7ffa5f374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:38:48.931016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf75ca0-5fff-467f-9f96-b7f7ffa5f374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:48.932658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf75ca0-5fff-467f-9f96-b7f7ffa5f374 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:48.948013 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1182/2351] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:38:48.952961 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaca4a9-4fd1-46fa-aa9b-38bcb4cf9cbb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:48.968610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaca4a9-4fd1-46fa-aa9b-38bcb4cf9cbb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:48.974865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc891a3-c69a-429d-ac7a-3186037d6039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:48.975564 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc891a3-c69a-429d-ac7a-3186037d6039 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:48.976673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc891a3-c69a-429d-ac7a-3186037d6039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:48.976819 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc891a3-c69a-429d-ac7a-3186037d6039 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:48.977972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc891a3-c69a-429d-ac7a-3186037d6039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:48.980199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc891a3-c69a-429d-ac7a-3186037d6039 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:49.022210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf75ca0-5fff-467f-9f96-b7f7ffa5f374 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:49.029913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1170/2352] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:38:48 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:38:49.035271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf75ca0-5fff-467f-9f96-b7f7ffa5f374 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:49.102786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9738cd4-a867-4b05-8ea2-7fa7209edbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:49.104430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9738cd4-a867-4b05-8ea2-7fa7209edbcc admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:49.104430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9738cd4-a867-4b05-8ea2-7fa7209edbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:49.104430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9738cd4-a867-4b05-8ea2-7fa7209edbcc admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:49.105230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9738cd4-a867-4b05-8ea2-7fa7209edbcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:49.107906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9738cd4-a867-4b05-8ea2-7fa7209edbcc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:49.148808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1183/2353] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:38:49 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:38:49.151916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc891a3-c69a-429d-ac7a-3186037d6039 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:49.164760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc891a3-c69a-429d-ac7a-3186037d6039 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:49.174020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9eb38-4d9b-4598-91e2-4abb1aa2c5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:49.174841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9eb38-4d9b-4598-91e2-4abb1aa2c5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:49.175370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9eb38-4d9b-4598-91e2-4abb1aa2c5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:49.175493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9eb38-4d9b-4598-91e2-4abb1aa2c5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:49.176322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9eb38-4d9b-4598-91e2-4abb1aa2c5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:49.178144 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9eb38-4d9b-4598-91e2-4abb1aa2c5c0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:49.192560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1171/2354] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:38:49.196301 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9738cd4-a867-4b05-8ea2-7fa7209edbcc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:49.207382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9738cd4-a867-4b05-8ea2-7fa7209edbcc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:49.213222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43e072-5521-4610-905e-35c6b7241481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:49.214045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43e072-5521-4610-905e-35c6b7241481 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:49.214430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43e072-5521-4610-905e-35c6b7241481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:49.215194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43e072-5521-4610-905e-35c6b7241481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:49.216005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43e072-5521-4610-905e-35c6b7241481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:49.218379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43e072-5521-4610-905e-35c6b7241481 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:49.242539 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1184/2355] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:38:49.246572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9eb38-4d9b-4598-91e2-4abb1aa2c5c0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:49.258246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9eb38-4d9b-4598-91e2-4abb1aa2c5c0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:49.264082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286e7c5-9993-4c4e-874a-64d9c611c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:49.265267 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286e7c5-9993-4c4e-874a-64d9c611c228 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:49.265549 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286e7c5-9993-4c4e-874a-64d9c611c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:49.265823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286e7c5-9993-4c4e-874a-64d9c611c228 admin admin] PATH_INFO: `/v3/projects/d458866e43c842d0877d4b59600a650a/users/07bb66529e6e4367bda80a65bf16dc02/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:49.285077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286e7c5-9993-4c4e-874a-64d9c611c228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d458866e43c842d0877d4b59600a650a, user_id=07bb66529e6e4367bda80a65bf16dc02, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:49.288266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286e7c5-9993-4c4e-874a-64d9c611c228 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:49.306277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1172/2356] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:38:49 2022] PUT /identity/v3/projects/d458866e43c842d0877d4b59600a650a/users/07bb66529e6e4367bda80a65bf16dc02/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:38:49.318563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f67c8a-4974-43e8-ba02-1e4199bfb82c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:49.318679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f67c8a-4974-43e8-ba02-1e4199bfb82c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:49.318900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f67c8a-4974-43e8-ba02-1e4199bfb82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:49.363480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-74f67c8a-4974-43e8-ba02-1e4199bfb82c None None] MFA Rules not processed for user `07bb66529e6e4367bda80a65bf16dc02`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:49.368210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-74f67c8a-4974-43e8-ba02-1e4199bfb82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:49.472240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1185/2357] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:49.485248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286e7c5-9993-4c4e-874a-64d9c611c228 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:49.487387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b286e7c5-9993-4c4e-874a-64d9c611c228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:49.589279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:49.642510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286e7c5-9993-4c4e-874a-64d9c611c228 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:49.646717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfa36b-2dfc-481f-ab98-795fe971e59b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:49.647983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfa36b-2dfc-481f-ab98-795fe971e59b service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:49.648465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfa36b-2dfc-481f-ab98-795fe971e59b service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:49.648964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfa36b-2dfc-481f-ab98-795fe971e59b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:49.663455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfa36b-2dfc-481f-ab98-795fe971e59b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:49.665236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfa36b-2dfc-481f-ab98-795fe971e59b service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:49.677854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1173/2358] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:38:50.508782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092ae82-9db6-468d-aecd-3e67a32ee37c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:50.511178 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092ae82-9db6-468d-aecd-3e67a32ee37c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:50.511923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092ae82-9db6-468d-aecd-3e67a32ee37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:50.527640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:38:50.547592 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6092ae82-9db6-468d-aecd-3e67a32ee37c None None] MFA Rules not processed for user `c77c4c5e840444929f648cfb735e995f`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:38:50.549522 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6092ae82-9db6-468d-aecd-3e67a32ee37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:50.632177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1186/2359] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:38:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:38:50.647213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfa36b-2dfc-481f-ab98-795fe971e59b service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:38:50.648795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-62dfa36b-2dfc-481f-ab98-795fe971e59b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:50.810912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfa36b-2dfc-481f-ab98-795fe971e59b service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:38:50.818155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:38:50.818909 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:38:50.819261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:38:50.819600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:38:50.821514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:38:50.898554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:38:50.900246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:38:50.910713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1174/2360] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:38:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:39:00.628987 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=31 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:00.906510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=82 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:03.649199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e75bc-f22e-4460-903f-2de146e6379d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:03.649729 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e75bc-f22e-4460-903f-2de146e6379d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:03.649834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e75bc-f22e-4460-903f-2de146e6379d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:03.699256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fc2e75bc-f22e-4460-903f-2de146e6379d None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:03.700528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fc2e75bc-f22e-4460-903f-2de146e6379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:03.770997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1187/2361] 10.222.0.69 () {58 vars in 967 bytes} [Fri Jun 24 10:39:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:03.793555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:03.795110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:03.894368 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba2c61-7ff2-4306-8957-1fc09d4bafbe service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:03.899828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020617ee-101d-4245-8350-1a1f83b91657 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:03.900476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020617ee-101d-4245-8350-1a1f83b91657 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:03.900761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020617ee-101d-4245-8350-1a1f83b91657 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:03.901053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020617ee-101d-4245-8350-1a1f83b91657 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:03.909623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020617ee-101d-4245-8350-1a1f83b91657 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:03.911200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020617ee-101d-4245-8350-1a1f83b91657 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:03.919213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1175/2362] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:39:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:39:13.767705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=40 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:13.915991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=41 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:14.626984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6047bb-2bb1-479a-8b1f-c89567d1f0fa None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:14.630855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6047bb-2bb1-479a-8b1f-c89567d1f0fa None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:14.631162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6047bb-2bb1-479a-8b1f-c89567d1f0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:14.654815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ff6047bb-2bb1-479a-8b1f-c89567d1f0fa None None] MFA Rules not processed for user `5fc073d480884782b4084952345382b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:14.657223 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ff6047bb-2bb1-479a-8b1f-c89567d1f0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:14.664613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1188/2363] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:24.645543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:25.356679 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4c0b7-69cf-4821-9780-d9fbe6c0dd47 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:25.357724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4c0b7-69cf-4821-9780-d9fbe6c0dd47 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:25.358106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4c0b7-69cf-4821-9780-d9fbe6c0dd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:25.376954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4fc4c0b7-69cf-4821-9780-d9fbe6c0dd47 None None] MFA Rules not processed for user `60367bf464f4441483bfe4a1373a1e92`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:25.378191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4fc4c0b7-69cf-4821-9780-d9fbe6c0dd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:25.437245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1176/2364] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:34.563149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf2a4-9094-4800-85b2-d127f8677f9c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:34.564180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf2a4-9094-4800-85b2-d127f8677f9c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:34.564663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf2a4-9094-4800-85b2-d127f8677f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:34.583897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cf1cf2a4-9094-4800-85b2-d127f8677f9c None None] MFA Rules not processed for user `960b18fe180f4185b291da1d9c18a2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:34.585397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cf1cf2a4-9094-4800-85b2-d127f8677f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:34.644945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1189/2365] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:34.823339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4c0b7-69cf-4821-9780-d9fbe6c0dd47 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:34.902729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4c0b7-69cf-4821-9780-d9fbe6c0dd47 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:34.911927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:34.913050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:34.913430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:34.913802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] PATH_INFO: `/v3/users/07bb66529e6e4367bda80a65bf16dc02` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:34.915309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07bb66529e6e4367bda80a65bf16dc02)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:34.919945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:34.983243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07bb66529e6e4367bda80a65bf16dc02'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.001175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07bb66529e6e4367bda80a65bf16dc02'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.011920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07bb66529e6e4367bda80a65bf16dc02'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.021332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07bb66529e6e4367bda80a65bf16dc02'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.022078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07bb66529e6e4367bda80a65bf16dc02'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.033165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1177/2366] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:39:34 2022] DELETE /identity/v3/users/07bb66529e6e4367bda80a65bf16dc02 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:39:35.035000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9853696-baaa-4893-95b4-5e464f5cf7bd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:35.035293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9853696-baaa-4893-95b4-5e464f5cf7bd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:35.035804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9853696-baaa-4893-95b4-5e464f5cf7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:35.064663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d9853696-baaa-4893-95b4-5e464f5cf7bd None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:35.065930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d9853696-baaa-4893-95b4-5e464f5cf7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:35.142715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1190/2367] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:35.151091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:35.152561 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:35.200105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:35.236184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80897c07-5313-428a-bddc-cd27d4920b18 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:35.242469 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184715c-e363-4b30-b681-516d4b69b95b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:35.243139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184715c-e363-4b30-b681-516d4b69b95b service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:35.243429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184715c-e363-4b30-b681-516d4b69b95b service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:35.243708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184715c-e363-4b30-b681-516d4b69b95b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:35.253729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184715c-e363-4b30-b681-516d4b69b95b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:35.255176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184715c-e363-4b30-b681-516d4b69b95b service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:35.264983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1178/2368] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:39:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:39:35.871472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9853696-baaa-4893-95b4-5e464f5cf7bd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:35.873170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d9853696-baaa-4893-95b4-5e464f5cf7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:35.885694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9853696-baaa-4893-95b4-5e464f5cf7bd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:35.891351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:35.892441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:35.892827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:35.893170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] PATH_INFO: `/v3/projects/d458866e43c842d0877d4b59600a650a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:35.894635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] RBAC: Authorizing `identity:delete_project(project_id=d458866e43c842d0877d4b59600a650a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:35.896645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:35.954778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd458866e43c842d0877d4b59600a650a', 'request_id': 'req-8b50f589-d15f-4a74-932d-1e11689220be'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.964325 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd458866e43c842d0877d4b59600a650a', 'request_id': 'req-8b50f589-d15f-4a74-932d-1e11689220be'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.970418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd458866e43c842d0877d4b59600a650a', 'request_id': 'req-8b50f589-d15f-4a74-932d-1e11689220be'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.981857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] The token cache is being invalidate because project d458866e43c842d0877d4b59600a650a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:39:35.982183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.982875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8b50f589-d15f-4a74-932d-1e11689220be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:35.984453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1191/2369] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:39:35 2022] DELETE /identity/v3/projects/d458866e43c842d0877d4b59600a650a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:39:35.991421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa0927-560b-498f-beb8-a96f3ba88903 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:35.991944 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa0927-560b-498f-beb8-a96f3ba88903 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:35.992058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa0927-560b-498f-beb8-a96f3ba88903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.017042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-abfa0927-560b-498f-beb8-a96f3ba88903 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:36.018246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-abfa0927-560b-498f-beb8-a96f3ba88903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:36.080984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1179/2370] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:36.089059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ce991-ac39-4d05-93c0-706df4ed2dd4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.089245 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ce991-ac39-4d05-93c0-706df4ed2dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.089545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ce991-ac39-4d05-93c0-706df4ed2dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.110082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-969ce991-ac39-4d05-93c0-706df4ed2dd4 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:36.111302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-969ce991-ac39-4d05-93c0-706df4ed2dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:36.120536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1192/2371] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:36.124838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa0927-560b-498f-beb8-a96f3ba88903 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:36.135171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa0927-560b-498f-beb8-a96f3ba88903 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:36.140493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9aae-1134-4cfa-8c7b-b4e52e44ae6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:36.141197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9aae-1134-4cfa-8c7b-b4e52e44ae6b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.141476 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9aae-1134-4cfa-8c7b-b4e52e44ae6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.141748 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9aae-1134-4cfa-8c7b-b4e52e44ae6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.142508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9aae-1134-4cfa-8c7b-b4e52e44ae6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:39:36.142835 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9aae-1134-4cfa-8c7b-b4e52e44ae6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:36.144465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9aae-1134-4cfa-8c7b-b4e52e44ae6b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:36.156907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1180/2372] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:39:36.160887 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ce991-ac39-4d05-93c0-706df4ed2dd4 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:36.170839 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ce991-ac39-4d05-93c0-706df4ed2dd4 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:36.176567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac40e72-c477-4bbc-94a8-047e882b258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:36.177357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac40e72-c477-4bbc-94a8-047e882b258b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.177710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac40e72-c477-4bbc-94a8-047e882b258b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.177973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac40e72-c477-4bbc-94a8-047e882b258b admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.179181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac40e72-c477-4bbc-94a8-047e882b258b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:36.181442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac40e72-c477-4bbc-94a8-047e882b258b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:36.205777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:36.219792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1193/2373] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:39:36 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:39:36.221222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9aae-1134-4cfa-8c7b-b4e52e44ae6b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:36.232798 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9aae-1134-4cfa-8c7b-b4e52e44ae6b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:36.295808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9004b-a77d-4ef2-a033-0708e910a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:36.296506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9004b-a77d-4ef2-a033-0708e910a898 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.296820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9004b-a77d-4ef2-a033-0708e910a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.297097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9004b-a77d-4ef2-a033-0708e910a898 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.297816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9004b-a77d-4ef2-a033-0708e910a898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:36.300618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9004b-a77d-4ef2-a033-0708e910a898 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:36.322334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1181/2374] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:39:36 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:39:36.327266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac40e72-c477-4bbc-94a8-047e882b258b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:36.338778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac40e72-c477-4bbc-94a8-047e882b258b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:36.343966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bf25f-6c00-46af-a1b7-a316213bc5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:36.344564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bf25f-6c00-46af-a1b7-a316213bc5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.344885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bf25f-6c00-46af-a1b7-a316213bc5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.345166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bf25f-6c00-46af-a1b7-a316213bc5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.345903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bf25f-6c00-46af-a1b7-a316213bc5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:36.347510 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bf25f-6c00-46af-a1b7-a316213bc5e7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:36.358788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1194/2375] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:39:36.363055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9004b-a77d-4ef2-a033-0708e910a898 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:36.373246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9004b-a77d-4ef2-a033-0708e910a898 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:36.377937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b655bc-afda-4056-a358-e826e3a273de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:36.378597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b655bc-afda-4056-a358-e826e3a273de admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.378883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b655bc-afda-4056-a358-e826e3a273de admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.379154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b655bc-afda-4056-a358-e826e3a273de admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.379883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b655bc-afda-4056-a358-e826e3a273de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:36.381520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b655bc-afda-4056-a358-e826e3a273de admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:36.391562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1182/2376] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:39:36.396408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bf25f-6c00-46af-a1b7-a316213bc5e7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:36.408247 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bf25f-6c00-46af-a1b7-a316213bc5e7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:36.413409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e003-5f33-4e3a-90e6-7bacf3a07c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:36.414288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e003-5f33-4e3a-90e6-7bacf3a07c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.414572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e003-5f33-4e3a-90e6-7bacf3a07c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.414844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e003-5f33-4e3a-90e6-7bacf3a07c9d admin admin] PATH_INFO: `/v3/projects/c1a98b2b2df84c7abc5b34d5431030d3/users/3b55a5e3b6b3453cac66b9599cef9ef7/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.430319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376e003-5f33-4e3a-90e6-7bacf3a07c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a98b2b2df84c7abc5b34d5431030d3, user_id=3b55a5e3b6b3453cac66b9599cef9ef7, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:36.433261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376e003-5f33-4e3a-90e6-7bacf3a07c9d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:36.448431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1195/2377] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:39:36 2022] PUT /identity/v3/projects/c1a98b2b2df84c7abc5b34d5431030d3/users/3b55a5e3b6b3453cac66b9599cef9ef7/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:39:36.459652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2930-af0d-4aa1-bb2a-9e31b42b9eae None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.459866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2930-af0d-4aa1-bb2a-9e31b42b9eae None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.460138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2930-af0d-4aa1-bb2a-9e31b42b9eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.489270 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1e2d2930-af0d-4aa1-bb2a-9e31b42b9eae None None] MFA Rules not processed for user `3b55a5e3b6b3453cac66b9599cef9ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:36.490527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1e2d2930-af0d-4aa1-bb2a-9e31b42b9eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:36.518163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:36.563504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1183/2378] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:36.575920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e003-5f33-4e3a-90e6-7bacf3a07c9d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:36.577764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4376e003-5f33-4e3a-90e6-7bacf3a07c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:36.639812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e003-5f33-4e3a-90e6-7bacf3a07c9d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:36.643935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e955c-458c-477c-8b8c-857456a4428e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:36.644593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e955c-458c-477c-8b8c-857456a4428e service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.644924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e955c-458c-477c-8b8c-857456a4428e service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.645207 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e955c-458c-477c-8b8c-857456a4428e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.654485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e955c-458c-477c-8b8c-857456a4428e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:36.655949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e955c-458c-477c-8b8c-857456a4428e service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:36.672580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1196/2379] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:39:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:39:36.699997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d2930-af0d-4aa1-bb2a-9e31b42b9eae None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:36.768140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d2930-af0d-4aa1-bb2a-9e31b42b9eae None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:36.772596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:36.773531 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.773810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.774082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] PATH_INFO: `/v3/users/3b55a5e3b6b3453cac66b9599cef9ef7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.775395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b55a5e3b6b3453cac66b9599cef9ef7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:36.777607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:36.833111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b55a5e3b6b3453cac66b9599cef9ef7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:36.846530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b55a5e3b6b3453cac66b9599cef9ef7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:36.859859 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b55a5e3b6b3453cac66b9599cef9ef7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:36.869053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b55a5e3b6b3453cac66b9599cef9ef7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:36.870161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b55a5e3b6b3453cac66b9599cef9ef7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:36.877810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1184/2380] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:39:36 2022] DELETE /identity/v3/users/3b55a5e3b6b3453cac66b9599cef9ef7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:39:36.882145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90be04-11e2-4273-a6cc-a83bcc556484 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:36.882966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90be04-11e2-4273-a6cc-a83bcc556484 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:36.883391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90be04-11e2-4273-a6cc-a83bcc556484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:36.902039 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2d90be04-11e2-4273-a6cc-a83bcc556484 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:36.903322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2d90be04-11e2-4273-a6cc-a83bcc556484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:36.986272 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1197/2381] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:36.996073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:36.997682 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:37.043222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:37.056885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701216f-eff0-4455-91b1-a0c44d68a8ea admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:37.061453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd7fd0-3a31-4592-8fd8-f94250a9bae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:37.062181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd7fd0-3a31-4592-8fd8-f94250a9bae8 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.062642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd7fd0-3a31-4592-8fd8-f94250a9bae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.063018 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd7fd0-3a31-4592-8fd8-f94250a9bae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.077719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd7fd0-3a31-4592-8fd8-f94250a9bae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:37.079186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd7fd0-3a31-4592-8fd8-f94250a9bae8 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:37.088854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1185/2382] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:39:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:39:37.407965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90be04-11e2-4273-a6cc-a83bcc556484 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:37.409493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2d90be04-11e2-4273-a6cc-a83bcc556484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:37.423599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90be04-11e2-4273-a6cc-a83bcc556484 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:37.431661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:37.432961 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.433496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.434415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] PATH_INFO: `/v3/projects/c1a98b2b2df84c7abc5b34d5431030d3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.435871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a98b2b2df84c7abc5b34d5431030d3)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:37.440869 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:37.520508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a98b2b2df84c7abc5b34d5431030d3', 'request_id': 'req-d617da01-255b-4954-9978-413bded76c02'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:37.530910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a98b2b2df84c7abc5b34d5431030d3', 'request_id': 'req-d617da01-255b-4954-9978-413bded76c02'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:37.535988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:37.537552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a98b2b2df84c7abc5b34d5431030d3', 'request_id': 'req-d617da01-255b-4954-9978-413bded76c02'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:37.550544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] The token cache is being invalidate because project c1a98b2b2df84c7abc5b34d5431030d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:39:37.550835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:37.551485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d617da01-255b-4954-9978-413bded76c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:37.553149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1198/2383] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:39:37 2022] DELETE /identity/v3/projects/c1a98b2b2df84c7abc5b34d5431030d3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:39:37.558172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f120421-b8c8-4804-a987-1e27282ea3ea None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.558483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f120421-b8c8-4804-a987-1e27282ea3ea None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.558760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f120421-b8c8-4804-a987-1e27282ea3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.576190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3f120421-b8c8-4804-a987-1e27282ea3ea None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:37.577460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3f120421-b8c8-4804-a987-1e27282ea3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:37.637132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1186/2384] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:39:37.647994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89d0c-fbd8-47d7-835d-baca7051212c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.648286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89d0c-fbd8-47d7-835d-baca7051212c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.648565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89d0c-fbd8-47d7-835d-baca7051212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.668362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4ea89d0c-fbd8-47d7-835d-baca7051212c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:37.669607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4ea89d0c-fbd8-47d7-835d-baca7051212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:37.679101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1199/2385] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:37.684228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f120421-b8c8-4804-a987-1e27282ea3ea None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:37.697955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f120421-b8c8-4804-a987-1e27282ea3ea None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:37.703782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b730eee-f86c-429b-8bda-0cd1152c58ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:37.704463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b730eee-f86c-429b-8bda-0cd1152c58ef admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.704763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b730eee-f86c-429b-8bda-0cd1152c58ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.705043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b730eee-f86c-429b-8bda-0cd1152c58ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.705766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b730eee-f86c-429b-8bda-0cd1152c58ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:39:37.706093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b730eee-f86c-429b-8bda-0cd1152c58ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:37.707669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b730eee-f86c-429b-8bda-0cd1152c58ef admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:37.720951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1187/2386] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:39:37.725427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea89d0c-fbd8-47d7-835d-baca7051212c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:37.736120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea89d0c-fbd8-47d7-835d-baca7051212c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:37.741707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ef119-bdf6-4ec0-85b1-216bafe5ba79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:37.742369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef119-bdf6-4ec0-85b1-216bafe5ba79 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.742675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef119-bdf6-4ec0-85b1-216bafe5ba79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.742922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef119-bdf6-4ec0-85b1-216bafe5ba79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.743612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ef119-bdf6-4ec0-85b1-216bafe5ba79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:37.745612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ef119-bdf6-4ec0-85b1-216bafe5ba79 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:37.775657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1200/2387] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:39:37 2022] POST /identity/v3/projects => generated 397 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:39:37.779978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b730eee-f86c-429b-8bda-0cd1152c58ef admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:37.792314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b730eee-f86c-429b-8bda-0cd1152c58ef admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:37.860826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acce2f0-e208-40ee-af03-d2710073f8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:37.861495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acce2f0-e208-40ee-af03-d2710073f8ec admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.861775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acce2f0-e208-40ee-af03-d2710073f8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.862044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acce2f0-e208-40ee-af03-d2710073f8ec admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.862731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acce2f0-e208-40ee-af03-d2710073f8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:37.864650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acce2f0-e208-40ee-af03-d2710073f8ec admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:37.895238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1188/2388] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:39:37 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:39:37.896464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ef119-bdf6-4ec0-85b1-216bafe5ba79 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:37.907351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ef119-bdf6-4ec0-85b1-216bafe5ba79 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:37.912385 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84aab9-4034-4a09-a708-c051d31dadb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:37.913091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84aab9-4034-4a09-a708-c051d31dadb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.913371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84aab9-4034-4a09-a708-c051d31dadb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.913651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84aab9-4034-4a09-a708-c051d31dadb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.914383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84aab9-4034-4a09-a708-c051d31dadb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:37.915994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84aab9-4034-4a09-a708-c051d31dadb9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:37.927160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1201/2389] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:39:37.930565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acce2f0-e208-40ee-af03-d2710073f8ec admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:37.941046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acce2f0-e208-40ee-af03-d2710073f8ec admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:37.945756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7c36d-d99e-4fc1-96e4-abe17d6482ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:37.946410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7c36d-d99e-4fc1-96e4-abe17d6482ef admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.946692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7c36d-d99e-4fc1-96e4-abe17d6482ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.946953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7c36d-d99e-4fc1-96e4-abe17d6482ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.947785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7c36d-d99e-4fc1-96e4-abe17d6482ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:37.949464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7c36d-d99e-4fc1-96e4-abe17d6482ef admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:37.959625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1189/2390] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:39:37.963794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84aab9-4034-4a09-a708-c051d31dadb9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:37.974099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84aab9-4034-4a09-a708-c051d31dadb9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:37.979549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c573f4-31f5-45b0-97ac-3661151cbbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:37.980441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c573f4-31f5-45b0-97ac-3661151cbbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:37.980722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c573f4-31f5-45b0-97ac-3661151cbbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:37.981032 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c573f4-31f5-45b0-97ac-3661151cbbe2 admin admin] PATH_INFO: `/v3/projects/395147d3b18b4a5f895639e2b9fe9987/users/8f8caacf74da41769c496121719118db/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:37.999891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c573f4-31f5-45b0-97ac-3661151cbbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395147d3b18b4a5f895639e2b9fe9987, user_id=8f8caacf74da41769c496121719118db, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:38.002610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c573f4-31f5-45b0-97ac-3661151cbbe2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:38.019284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1202/2391] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:39:37 2022] PUT /identity/v3/projects/395147d3b18b4a5f895639e2b9fe9987/users/8f8caacf74da41769c496121719118db/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:39:38.032163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53cef8-f159-4bb8-8ce7-199c31ca4329 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:38.032703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53cef8-f159-4bb8-8ce7-199c31ca4329 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:38.033106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53cef8-f159-4bb8-8ce7-199c31ca4329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:38.069861 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-eb53cef8-f159-4bb8-8ce7-199c31ca4329 None None] MFA Rules not processed for user `8f8caacf74da41769c496121719118db`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:38.071174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eb53cef8-f159-4bb8-8ce7-199c31ca4329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:38.121068 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:38.178206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1190/2392] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:39:38.190267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c573f4-31f5-45b0-97ac-3661151cbbe2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:38.192072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d8c573f4-31f5-45b0-97ac-3661151cbbe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:38.251816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c573f4-31f5-45b0-97ac-3661151cbbe2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:38.255665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448bf3d-6e74-4bfe-abe0-686a6b7a4676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:38.256319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448bf3d-6e74-4bfe-abe0-686a6b7a4676 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:38.256601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448bf3d-6e74-4bfe-abe0-686a6b7a4676 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:38.256962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448bf3d-6e74-4bfe-abe0-686a6b7a4676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:38.266615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448bf3d-6e74-4bfe-abe0-686a6b7a4676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:38.268059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448bf3d-6e74-4bfe-abe0-686a6b7a4676 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:38.278470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1203/2393] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:39:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:39:41.409504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53cef8-f159-4bb8-8ce7-199c31ca4329 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:41.410899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eb53cef8-f159-4bb8-8ce7-199c31ca4329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:41.483860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53cef8-f159-4bb8-8ce7-199c31ca4329 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:41.488567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226d61a-0c46-434b-b333-230f565e93f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:41.489261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226d61a-0c46-434b-b333-230f565e93f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:41.489551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226d61a-0c46-434b-b333-230f565e93f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:41.489856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226d61a-0c46-434b-b333-230f565e93f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:41.492015 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2226d61a-0c46-434b-b333-230f565e93f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:41.558289 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226d61a-0c46-434b-b333-230f565e93f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:41.559737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226d61a-0c46-434b-b333-230f565e93f9 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:41.569779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1191/2394] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:39:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:39:46.418389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448bf3d-6e74-4bfe-abe0-686a6b7a4676 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:46.490296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448bf3d-6e74-4bfe-abe0-686a6b7a4676 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:46.495292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:46.496168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:46.496455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:46.496757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] PATH_INFO: `/v3/users/8f8caacf74da41769c496121719118db` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:46.498515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8caacf74da41769c496121719118db)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:46.500463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:46.543691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:46.557816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8caacf74da41769c496121719118db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:46.571353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8caacf74da41769c496121719118db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:46.581735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8caacf74da41769c496121719118db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:46.582500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8caacf74da41769c496121719118db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:46.589453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8caacf74da41769c496121719118db'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:46.596287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1204/2395] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:39:46 2022] DELETE /identity/v3/users/8f8caacf74da41769c496121719118db => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:39:46.601290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887f69a-2019-4ea9-9a51-f26279058835 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:46.601800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887f69a-2019-4ea9-9a51-f26279058835 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:46.601899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887f69a-2019-4ea9-9a51-f26279058835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:46.619765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9887f69a-2019-4ea9-9a51-f26279058835 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:39:46.621016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9887f69a-2019-4ea9-9a51-f26279058835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:46.684640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1192/2396] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:39:46.694770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:46.696360 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:46.765449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294c928-23c3-49d4-906c-98289e7e7190 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:46.771467 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcf8d0-864d-45ac-b9d7-35cb2c00679e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:46.772429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcf8d0-864d-45ac-b9d7-35cb2c00679e service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:46.772847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcf8d0-864d-45ac-b9d7-35cb2c00679e service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:46.773278 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcf8d0-864d-45ac-b9d7-35cb2c00679e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:46.784831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcf8d0-864d-45ac-b9d7-35cb2c00679e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:46.786274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcf8d0-864d-45ac-b9d7-35cb2c00679e service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:46.795908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1205/2397] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:39:47.078591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887f69a-2019-4ea9-9a51-f26279058835 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:39:47.080290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9887f69a-2019-4ea9-9a51-f26279058835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:39:47.095055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887f69a-2019-4ea9-9a51-f26279058835 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:39:47.100502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:39:47.101414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:39:47.101699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:39:47.101973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] PATH_INFO: `/v3/projects/395147d3b18b4a5f895639e2b9fe9987` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:39:47.103261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=395147d3b18b4a5f895639e2b9fe9987)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:39:47.105695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:39:47.141544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:47.159086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '395147d3b18b4a5f895639e2b9fe9987', 'request_id': 'req-73f3d1b0-fa24-4997-9134-c56050f9ccf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:47.165187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '395147d3b18b4a5f895639e2b9fe9987', 'request_id': 'req-73f3d1b0-fa24-4997-9134-c56050f9ccf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:47.175722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '395147d3b18b4a5f895639e2b9fe9987', 'request_id': 'req-73f3d1b0-fa24-4997-9134-c56050f9ccf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:47.187364 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] The token cache is being invalidate because project 395147d3b18b4a5f895639e2b9fe9987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:39:47.187600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:47.188254 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:39:47.189938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1193/2398] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:39:47 2022] DELETE /identity/v3/projects/395147d3b18b4a5f895639e2b9fe9987 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:39:56.791526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=43,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:39:57.183882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:08.317014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cad50-61bb-4a56-965c-ac4d33f5e092 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:08.317668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cad50-61bb-4a56-965c-ac4d33f5e092 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:08.317832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cad50-61bb-4a56-965c-ac4d33f5e092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:08.335817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f39cad50-61bb-4a56-965c-ac4d33f5e092 None None] MFA Rules not processed for user `3e4ed08c1c81453abaa6c1dd826f0258`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:40:08.337076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f39cad50-61bb-4a56-965c-ac4d33f5e092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:08.392647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1206/2399] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:40:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:40:18.386001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=25 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:24.939839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:24.942279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:25.008174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3d1b0-fa24-4997-9134-c56050f9ccf2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:25.012409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951535fb-2756-4dcd-980b-b80db073f6a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:25.013063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951535fb-2756-4dcd-980b-b80db073f6a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:25.013345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951535fb-2756-4dcd-980b-b80db073f6a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:25.013607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951535fb-2756-4dcd-980b-b80db073f6a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:25.015602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-951535fb-2756-4dcd-980b-b80db073f6a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:25.097938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951535fb-2756-4dcd-980b-b80db073f6a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:25.099391 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951535fb-2756-4dcd-980b-b80db073f6a9 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:25.110869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1194/2400] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:40:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:40:25.379212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cad50-61bb-4a56-965c-ac4d33f5e092 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:25.389285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cad50-61bb-4a56-965c-ac4d33f5e092 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:25.393517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:25.394215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:25.394493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:25.394775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:25.396606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:25.410621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:25.412083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:25.423542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1207/2401] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:40:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:40:31.273455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c4cd6-139d-4f67-81d1-a4acaf8b1252 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:31.273940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c4cd6-139d-4f67-81d1-a4acaf8b1252 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:31.274093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c4cd6-139d-4f67-81d1-a4acaf8b1252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:31.291922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0c1c4cd6-139d-4f67-81d1-a4acaf8b1252 None None] MFA Rules not processed for user `733b7d5058da45089a2133e2fe4695e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:40:31.293125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0c1c4cd6-139d-4f67-81d1-a4acaf8b1252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:31.352999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1195/2402] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:40:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:40:32.943754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:32.945369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:33.017684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0a0e9-f206-40a7-b382-5667396a2473 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:33.022725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:33.023596 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:33.023881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:33.024329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] PATH_INFO: `/v3/users/60367bf464f4441483bfe4a1373a1e92` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:33.025641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60367bf464f4441483bfe4a1373a1e92)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:33.027496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:33.080758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60367bf464f4441483bfe4a1373a1e92'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.092631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60367bf464f4441483bfe4a1373a1e92'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.101586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60367bf464f4441483bfe4a1373a1e92'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.102537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60367bf464f4441483bfe4a1373a1e92'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.110023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60367bf464f4441483bfe4a1373a1e92'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.116852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1208/2403] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:40:32 2022] DELETE /identity/v3/users/60367bf464f4441483bfe4a1373a1e92 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:40:33.313796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c4cd6-139d-4f67-81d1-a4acaf8b1252 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:33.325139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c4cd6-139d-4f67-81d1-a4acaf8b1252 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:33.370862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:33.383929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:33.384976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:33.385247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:33.385526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] PATH_INFO: `/v3/projects/b88db10e9091496a84e87446da50a3b6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:33.386745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b88db10e9091496a84e87446da50a3b6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:33.388831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:33.449374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b88db10e9091496a84e87446da50a3b6', 'request_id': 'req-3c461f65-e636-4e70-972e-68aa53fb3f77'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.456371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b88db10e9091496a84e87446da50a3b6', 'request_id': 'req-3c461f65-e636-4e70-972e-68aa53fb3f77'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.467508 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b88db10e9091496a84e87446da50a3b6', 'request_id': 'req-3c461f65-e636-4e70-972e-68aa53fb3f77'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.479448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] The token cache is being invalidate because project b88db10e9091496a84e87446da50a3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:40:33.479732 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.480355 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:33.481963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1196/2404] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:40:33 2022] DELETE /identity/v3/projects/b88db10e9091496a84e87446da50a3b6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:40:34.165911 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:34.167625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:34.245607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:34.250728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f4add-26da-46d2-92c5-b9ad678d33d5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:34.319955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:34.320616 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:34.321062 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:34.321462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:34.323417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:34.381037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:34.382625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:34.392200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1209/2405] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:40:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:40:41.427143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:41.428856 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:41.495243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c461f65-e636-4e70-972e-68aa53fb3f77 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:41.499225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:41.499873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:41.500148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:41.500423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:41.502347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:41.575791 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:41.577289 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:41.587238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1197/2406] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:40:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:40:44.387500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=54 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:44.933165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:44.934580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:44.993333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:44.995368 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:45.009005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6639cc-bfa6-41b8-9986-6501dc032bab service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:45.014171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:45.015067 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.015357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.015802 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] PATH_INFO: `/v3/users/960b18fe180f4185b291da1d9c18a2fa` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.017054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=960b18fe180f4185b291da1d9c18a2fa)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:45.019003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:45.025030 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66c15c-d675-4d70-89b6-29951e52934a service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:45.032065 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fabdb5-afb0-4916-9a28-a263fa791387 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:45.033649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fabdb5-afb0-4916-9a28-a263fa791387 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.034133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fabdb5-afb0-4916-9a28-a263fa791387 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.034641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fabdb5-afb0-4916-9a28-a263fa791387 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.037760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-92fabdb5-afb0-4916-9a28-a263fa791387 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:45.052973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:45.063997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fabdb5-afb0-4916-9a28-a263fa791387 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:45.065877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fabdb5-afb0-4916-9a28-a263fa791387 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:45.080918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1210/2407] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:40:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:40:45.100711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '960b18fe180f4185b291da1d9c18a2fa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.118057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '960b18fe180f4185b291da1d9c18a2fa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.137365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '960b18fe180f4185b291da1d9c18a2fa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.149719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '960b18fe180f4185b291da1d9c18a2fa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.150533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d826673-bac6-4e12-9bf9-d758a1fd7f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '960b18fe180f4185b291da1d9c18a2fa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.157784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1198/2408] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:40:44 2022] DELETE /identity/v3/users/960b18fe180f4185b291da1d9c18a2fa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:40:45.350524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fabdb5-afb0-4916-9a28-a263fa791387 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:45.366035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fabdb5-afb0-4916-9a28-a263fa791387 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:45.430391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:45.431311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.431580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.431856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] PATH_INFO: `/v3/projects/d8cc166ec03349b3b2b0d69840886bd9` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.433193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8cc166ec03349b3b2b0d69840886bd9)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:45.435227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:45.487390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8cc166ec03349b3b2b0d69840886bd9', 'request_id': 'req-21e95af5-3ffa-4116-b132-3feafd7a6561'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.497175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8cc166ec03349b3b2b0d69840886bd9', 'request_id': 'req-21e95af5-3ffa-4116-b132-3feafd7a6561'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.502877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8cc166ec03349b3b2b0d69840886bd9', 'request_id': 'req-21e95af5-3ffa-4116-b132-3feafd7a6561'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.514305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] The token cache is being invalidate because project d8cc166ec03349b3b2b0d69840886bd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:40:45.514466 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.515205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-21e95af5-3ffa-4116-b132-3feafd7a6561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:40:45.516793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1211/2409] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:40:45 2022] DELETE /identity/v3/projects/d8cc166ec03349b3b2b0d69840886bd9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:40:45.522000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4ef4e-fda4-419a-b5c4-7c8f28229ef3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.522281 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4ef4e-fda4-419a-b5c4-7c8f28229ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.522551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4ef4e-fda4-419a-b5c4-7c8f28229ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.541046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b4c4ef4e-fda4-419a-b5c4-7c8f28229ef3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:40:45.542569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b4c4ef4e-fda4-419a-b5c4-7c8f28229ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:45.613386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1199/2410] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:40:45.622260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e46e8-6595-40fd-9e50-85c026614153 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.622552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e46e8-6595-40fd-9e50-85c026614153 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.622955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e46e8-6595-40fd-9e50-85c026614153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.646003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-647e46e8-6595-40fd-9e50-85c026614153 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:40:45.647246 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-647e46e8-6595-40fd-9e50-85c026614153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:45.657398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1212/2411] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:40:45.662078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4ef4e-fda4-419a-b5c4-7c8f28229ef3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:45.672505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4ef4e-fda4-419a-b5c4-7c8f28229ef3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:45.677796 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189ee42-1f81-491a-936e-f466d3ed88b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:45.678458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189ee42-1f81-491a-936e-f466d3ed88b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.678740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189ee42-1f81-491a-936e-f466d3ed88b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.679010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189ee42-1f81-491a-936e-f466d3ed88b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.679752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189ee42-1f81-491a-936e-f466d3ed88b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:40:45.680093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189ee42-1f81-491a-936e-f466d3ed88b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:45.681736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189ee42-1f81-491a-936e-f466d3ed88b3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:45.694262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1200/2412] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:40:45.698563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e46e8-6595-40fd-9e50-85c026614153 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:45.707944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e46e8-6595-40fd-9e50-85c026614153 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:45.712922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779b064-cfc0-4427-ac08-1d9c88c53a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:45.713567 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779b064-cfc0-4427-ac08-1d9c88c53a6b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.713845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779b064-cfc0-4427-ac08-1d9c88c53a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.714121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779b064-cfc0-4427-ac08-1d9c88c53a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.714806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779b064-cfc0-4427-ac08-1d9c88c53a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:45.716724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779b064-cfc0-4427-ac08-1d9c88c53a6b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:45.727053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:45.745937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1213/2413] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:40:45 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:40:45.750264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189ee42-1f81-491a-936e-f466d3ed88b3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:45.760528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189ee42-1f81-491a-936e-f466d3ed88b3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:45.828405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e62d1-6f3f-49a6-8ecc-5c22dd5b0e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:45.829019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e62d1-6f3f-49a6-8ecc-5c22dd5b0e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.829306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e62d1-6f3f-49a6-8ecc-5c22dd5b0e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.829581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e62d1-6f3f-49a6-8ecc-5c22dd5b0e63 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.830239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e62d1-6f3f-49a6-8ecc-5c22dd5b0e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:45.832223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e62d1-6f3f-49a6-8ecc-5c22dd5b0e63 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:45.855277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1201/2414] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:40:45 2022] POST /identity/v3/users => generated 278 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:40:45.858962 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779b064-cfc0-4427-ac08-1d9c88c53a6b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:45.869818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779b064-cfc0-4427-ac08-1d9c88c53a6b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:45.874977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c039d0-eaae-44c9-b66c-b1f080974625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:45.875640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c039d0-eaae-44c9-b66c-b1f080974625 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.875928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c039d0-eaae-44c9-b66c-b1f080974625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.876203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c039d0-eaae-44c9-b66c-b1f080974625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.876980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c039d0-eaae-44c9-b66c-b1f080974625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:45.878590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c039d0-eaae-44c9-b66c-b1f080974625 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:45.889520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1214/2415] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:40:45.893509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e62d1-6f3f-49a6-8ecc-5c22dd5b0e63 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:45.903747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e62d1-6f3f-49a6-8ecc-5c22dd5b0e63 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:45.908657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d492093-350f-4a7f-b10a-8673d522b1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:45.909344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d492093-350f-4a7f-b10a-8673d522b1ce admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.909631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d492093-350f-4a7f-b10a-8673d522b1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.909910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d492093-350f-4a7f-b10a-8673d522b1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.910671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d492093-350f-4a7f-b10a-8673d522b1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:45.912269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d492093-350f-4a7f-b10a-8673d522b1ce admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:45.922307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1202/2416] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:40:45.926316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c039d0-eaae-44c9-b66c-b1f080974625 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:45.936548 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c039d0-eaae-44c9-b66c-b1f080974625 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:45.941794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f623f-6cea-4cda-8418-60134844a4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:45.943150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f623f-6cea-4cda-8418-60134844a4a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.943434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f623f-6cea-4cda-8418-60134844a4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.943711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f623f-6cea-4cda-8418-60134844a4a0 admin admin] PATH_INFO: `/v3/projects/82c3be2fd4e74c24aec68aba519c9a74/users/1679609b7d434a2cb6e0e7092396d5c8/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:45.959203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f623f-6cea-4cda-8418-60134844a4a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c3be2fd4e74c24aec68aba519c9a74, user_id=1679609b7d434a2cb6e0e7092396d5c8, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:45.961950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f623f-6cea-4cda-8418-60134844a4a0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:45.978834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1215/2417] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:40:45 2022] PUT /identity/v3/projects/82c3be2fd4e74c24aec68aba519c9a74/users/1679609b7d434a2cb6e0e7092396d5c8/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:40:45.993821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3929e-ac43-4f14-81ea-f686e38d250e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:45.994322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3929e-ac43-4f14-81ea-f686e38d250e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:45.994417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3929e-ac43-4f14-81ea-f686e38d250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.002439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:46.024875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-66e3929e-ac43-4f14-81ea-f686e38d250e None None] MFA Rules not processed for user `1679609b7d434a2cb6e0e7092396d5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:40:46.026126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-66e3929e-ac43-4f14-81ea-f686e38d250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:46.095657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1203/2418] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:40:45 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:40:46.100266 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f623f-6cea-4cda-8418-60134844a4a0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:46.168227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f623f-6cea-4cda-8418-60134844a4a0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:46.173443 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf4085-379f-43d4-ac13-a8f7ba13d976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:46.174125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf4085-379f-43d4-ac13-a8f7ba13d976 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:46.174420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf4085-379f-43d4-ac13-a8f7ba13d976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:46.174701 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf4085-379f-43d4-ac13-a8f7ba13d976 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.175410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf4085-379f-43d4-ac13-a8f7ba13d976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:46.177368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf4085-379f-43d4-ac13-a8f7ba13d976 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:46.206081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1216/2419] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:40:46 2022] POST /identity/v3/projects => generated 355 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:40:46.210185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3929e-ac43-4f14-81ea-f686e38d250e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:46.223991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3929e-ac43-4f14-81ea-f686e38d250e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:46.291322 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbda156-a69f-4012-82af-3cbc378b9985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:46.292041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbda156-a69f-4012-82af-3cbc378b9985 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:46.292358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbda156-a69f-4012-82af-3cbc378b9985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:46.292604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbda156-a69f-4012-82af-3cbc378b9985 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.293340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbda156-a69f-4012-82af-3cbc378b9985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:46.295685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbda156-a69f-4012-82af-3cbc378b9985 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:46.317388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1204/2420] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:40:46 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:40:46.321408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf4085-379f-43d4-ac13-a8f7ba13d976 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:46.331943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf4085-379f-43d4-ac13-a8f7ba13d976 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:46.336617 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b4949-427c-4096-b36b-68088b538c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:46.337316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b4949-427c-4096-b36b-68088b538c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:46.337602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b4949-427c-4096-b36b-68088b538c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:46.337895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b4949-427c-4096-b36b-68088b538c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.338610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b4949-427c-4096-b36b-68088b538c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:46.340224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b4949-427c-4096-b36b-68088b538c4e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:46.352071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1217/2421] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:40:46.356300 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbda156-a69f-4012-82af-3cbc378b9985 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:46.369087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbda156-a69f-4012-82af-3cbc378b9985 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:46.373825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b1cd1-1e12-4021-b057-3eb85b05a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:46.374712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b1cd1-1e12-4021-b057-3eb85b05a287 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:46.374999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b1cd1-1e12-4021-b057-3eb85b05a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:46.375278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b1cd1-1e12-4021-b057-3eb85b05a287 admin admin] PATH_INFO: `/v3/projects/1f819bcd53ec42c1b55b6572c008c3c1/users/d1d0c63dc9664180b8b357b5629d819b/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.392491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b1cd1-1e12-4021-b057-3eb85b05a287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f819bcd53ec42c1b55b6572c008c3c1, user_id=d1d0c63dc9664180b8b357b5629d819b, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:46.395231 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b1cd1-1e12-4021-b057-3eb85b05a287 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:46.409805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1205/2422] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:40:46 2022] PUT /identity/v3/projects/1f819bcd53ec42c1b55b6572c008c3c1/users/d1d0c63dc9664180b8b357b5629d819b/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:40:46.414644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b4949-427c-4096-b36b-68088b538c4e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:46.425412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b4949-427c-4096-b36b-68088b538c4e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:46.485027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24175b96-ff72-4f79-a83a-2c9c51c43fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:46.485806 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24175b96-ff72-4f79-a83a-2c9c51c43fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:46.486092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24175b96-ff72-4f79-a83a-2c9c51c43fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:46.486383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24175b96-ff72-4f79-a83a-2c9c51c43fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.487276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24175b96-ff72-4f79-a83a-2c9c51c43fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:46.488924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24175b96-ff72-4f79-a83a-2c9c51c43fd1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:46.502769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1218/2423] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:40:46.506722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b1cd1-1e12-4021-b057-3eb85b05a287 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:46.517422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b1cd1-1e12-4021-b057-3eb85b05a287 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:46.522036 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07387f-c2ac-4cfd-b8f4-99220871ce65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:46.522932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07387f-c2ac-4cfd-b8f4-99220871ce65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:46.523211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07387f-c2ac-4cfd-b8f4-99220871ce65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:46.523482 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07387f-c2ac-4cfd-b8f4-99220871ce65 admin admin] PATH_INFO: `/v3/projects/1f819bcd53ec42c1b55b6572c008c3c1/users/d1d0c63dc9664180b8b357b5629d819b/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.525464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07387f-c2ac-4cfd-b8f4-99220871ce65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f819bcd53ec42c1b55b6572c008c3c1, user_id=d1d0c63dc9664180b8b357b5629d819b, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:46.528534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07387f-c2ac-4cfd-b8f4-99220871ce65 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:46.539388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1206/2424] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:40:46 2022] PUT /identity/v3/projects/1f819bcd53ec42c1b55b6572c008c3c1/users/d1d0c63dc9664180b8b357b5629d819b/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:40:46.551184 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c789-e0c9-4e25-8959-df61b024ed2a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:46.551476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c789-e0c9-4e25-8959-df61b024ed2a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:46.551759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c789-e0c9-4e25-8959-df61b024ed2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.588072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ccf5c789-e0c9-4e25-8959-df61b024ed2a None None] MFA Rules not processed for user `d1d0c63dc9664180b8b357b5629d819b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:40:46.589432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ccf5c789-e0c9-4e25-8959-df61b024ed2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:46.604339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:46.668817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1219/2425] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:40:46 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:40:46.680657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07387f-c2ac-4cfd-b8f4-99220871ce65 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:46.682453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2e07387f-c2ac-4cfd-b8f4-99220871ce65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:40:46.738682 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:46.745052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07387f-c2ac-4cfd-b8f4-99220871ce65 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:46.750257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78984089-6f41-4eb1-a4d8-2d5b387a1e1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:46.751048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78984089-6f41-4eb1-a4d8-2d5b387a1e1e service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:46.751331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78984089-6f41-4eb1-a4d8-2d5b387a1e1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:46.751638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78984089-6f41-4eb1-a4d8-2d5b387a1e1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.761457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78984089-6f41-4eb1-a4d8-2d5b387a1e1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:46.762971 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78984089-6f41-4eb1-a4d8-2d5b387a1e1e service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:46.772927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1207/2426] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:40:46 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:40:46.849162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c789-e0c9-4e25-8959-df61b024ed2a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:40:46.858854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c789-e0c9-4e25-8959-df61b024ed2a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:40:46.862322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366ae68-48c6-4de3-b20c-6161b32c1034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:40:46.862990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366ae68-48c6-4de3-b20c-6161b32c1034 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:40:46.863270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366ae68-48c6-4de3-b20c-6161b32c1034 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:40:46.863551 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366ae68-48c6-4de3-b20c-6161b32c1034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:40:46.922706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366ae68-48c6-4de3-b20c-6161b32c1034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:40:46.924188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366ae68-48c6-4de3-b20c-6161b32c1034 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:40:46.933324 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1220/2427] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:40:46 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:40:56.769052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:40:56.929075 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=50 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:02.141705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78984089-6f41-4eb1-a4d8-2d5b387a1e1e service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:02.206781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78984089-6f41-4eb1-a4d8-2d5b387a1e1e service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:02.211659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:02.212534 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:02.212843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:02.213093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] PATH_INFO: `/v3/users/1679609b7d434a2cb6e0e7092396d5c8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:02.214311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1679609b7d434a2cb6e0e7092396d5c8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:02.216229 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:02.268593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1679609b7d434a2cb6e0e7092396d5c8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.282730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1679609b7d434a2cb6e0e7092396d5c8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.293399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1679609b7d434a2cb6e0e7092396d5c8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.302380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1679609b7d434a2cb6e0e7092396d5c8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.303030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1679609b7d434a2cb6e0e7092396d5c8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.309502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1208/2428] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:02 2022] DELETE /identity/v3/users/1679609b7d434a2cb6e0e7092396d5c8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:02.314934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25f032-b2a0-457f-8cf2-6044209ed533 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:02.315518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25f032-b2a0-457f-8cf2-6044209ed533 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:02.315621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25f032-b2a0-457f-8cf2-6044209ed533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:02.333649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-aa25f032-b2a0-457f-8cf2-6044209ed533 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:02.334903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aa25f032-b2a0-457f-8cf2-6044209ed533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:02.402631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1221/2429] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:02.412573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:02.414149 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:02.476843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dee7be-d1c5-408a-a9bc-3c043cda22cc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:02.481235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8515b-2c1b-43e8-ad2a-d4e8006a624e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:02.481887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8515b-2c1b-43e8-ad2a-d4e8006a624e service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:02.482175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8515b-2c1b-43e8-ad2a-d4e8006a624e service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:02.482443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8515b-2c1b-43e8-ad2a-d4e8006a624e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:02.492558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8515b-2c1b-43e8-ad2a-d4e8006a624e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:02.494064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8515b-2c1b-43e8-ad2a-d4e8006a624e service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:02.505013 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1209/2430] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:02.787853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25f032-b2a0-457f-8cf2-6044209ed533 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:02.789355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aa25f032-b2a0-457f-8cf2-6044209ed533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:02.801683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25f032-b2a0-457f-8cf2-6044209ed533 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:02.806758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:02.807673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:02.807956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:02.808236 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] PATH_INFO: `/v3/projects/82c3be2fd4e74c24aec68aba519c9a74` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:02.809419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c3be2fd4e74c24aec68aba519c9a74)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:02.811558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:02.865501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c3be2fd4e74c24aec68aba519c9a74', 'request_id': 'req-a71510f8-15aa-48b8-8367-bd509e24c4fc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.874235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c3be2fd4e74c24aec68aba519c9a74', 'request_id': 'req-a71510f8-15aa-48b8-8367-bd509e24c4fc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.880107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c3be2fd4e74c24aec68aba519c9a74', 'request_id': 'req-a71510f8-15aa-48b8-8367-bd509e24c4fc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.891165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] The token cache is being invalidate because project 82c3be2fd4e74c24aec68aba519c9a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:02.891450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.892148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:02.893809 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1222/2431] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:02 2022] DELETE /identity/v3/projects/82c3be2fd4e74c24aec68aba519c9a74 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:02.897938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8515b-2c1b-43e8-ad2a-d4e8006a624e service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:02.907883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8515b-2c1b-43e8-ad2a-d4e8006a624e service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:02.929117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:02.972016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:02.973157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:02.973440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:02.973712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] PATH_INFO: `/v3/users/d1d0c63dc9664180b8b357b5629d819b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:02.975285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d0c63dc9664180b8b357b5629d819b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:02.977272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:03.028977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d0c63dc9664180b8b357b5629d819b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.040666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d0c63dc9664180b8b357b5629d819b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.050206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d0c63dc9664180b8b357b5629d819b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.057317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d0c63dc9664180b8b357b5629d819b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.058338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a7adf264-89f5-4aa4-a150-bf736b76a027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d0c63dc9664180b8b357b5629d819b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.066310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1210/2432] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:02 2022] DELETE /identity/v3/users/d1d0c63dc9664180b8b357b5629d819b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:03.356598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:03.358229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:03.373982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71510f8-15aa-48b8-8367-bd509e24c4fc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:03.435268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:03.436257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.436648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.437153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] PATH_INFO: `/v3/projects/1f819bcd53ec42c1b55b6572c008c3c1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:03.439199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f819bcd53ec42c1b55b6572c008c3c1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:03.442619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:03.479458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:03.497554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f819bcd53ec42c1b55b6572c008c3c1', 'request_id': 'req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.506716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f819bcd53ec42c1b55b6572c008c3c1', 'request_id': 'req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.512268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f819bcd53ec42c1b55b6572c008c3c1', 'request_id': 'req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.523404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] The token cache is being invalidate because project 1f819bcd53ec42c1b55b6572c008c3c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:03.523685 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.524300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c58b1bb2-212d-4f90-a33a-8ab463c59ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:03.525865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1223/2433] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:03 2022] DELETE /identity/v3/projects/1f819bcd53ec42c1b55b6572c008c3c1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:03.530983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db0336-bff2-4eff-b704-30b405a6c3da None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.531506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db0336-bff2-4eff-b704-30b405a6c3da None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.531605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db0336-bff2-4eff-b704-30b405a6c3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:03.550288 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a4db0336-bff2-4eff-b704-30b405a6c3da None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:03.551510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a4db0336-bff2-4eff-b704-30b405a6c3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:03.622470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1211/2434] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:03.630605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855179f-ca19-48ac-8c7f-1d23b63f0fd1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.630889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855179f-ca19-48ac-8c7f-1d23b63f0fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.631164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855179f-ca19-48ac-8c7f-1d23b63f0fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:03.649760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a855179f-ca19-48ac-8c7f-1d23b63f0fd1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:03.650958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a855179f-ca19-48ac-8c7f-1d23b63f0fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:03.659018 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1224/2435] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:03.663223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db0336-bff2-4eff-b704-30b405a6c3da None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:03.673169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db0336-bff2-4eff-b704-30b405a6c3da None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:03.678078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0002-b3b6-423b-9a21-c6b6522c4d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:03.678727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0002-b3b6-423b-9a21-c6b6522c4d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.679012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0002-b3b6-423b-9a21-c6b6522c4d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.679287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0002-b3b6-423b-9a21-c6b6522c4d27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:03.680023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac0002-b3b6-423b-9a21-c6b6522c4d27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:41:03.680331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac0002-b3b6-423b-9a21-c6b6522c4d27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:03.681962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac0002-b3b6-423b-9a21-c6b6522c4d27 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:03.694207 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1212/2436] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:41:03.698606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855179f-ca19-48ac-8c7f-1d23b63f0fd1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:03.709084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855179f-ca19-48ac-8c7f-1d23b63f0fd1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:03.714921 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a260de-d3f1-4670-ae07-83bd47df33cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:03.715571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a260de-d3f1-4670-ae07-83bd47df33cf admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.715851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a260de-d3f1-4670-ae07-83bd47df33cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.716120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a260de-d3f1-4670-ae07-83bd47df33cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:03.716825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a260de-d3f1-4670-ae07-83bd47df33cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:03.719085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a260de-d3f1-4670-ae07-83bd47df33cf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:03.747453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1225/2437] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:41:03 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:03.751509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0002-b3b6-423b-9a21-c6b6522c4d27 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:03.761629 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0002-b3b6-423b-9a21-c6b6522c4d27 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:03.788016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:03.821135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a7e98-6e0e-4421-9b5b-0ba9952376b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:03.821813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a7e98-6e0e-4421-9b5b-0ba9952376b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.822090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a7e98-6e0e-4421-9b5b-0ba9952376b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.822362 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a7e98-6e0e-4421-9b5b-0ba9952376b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:03.823047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a7e98-6e0e-4421-9b5b-0ba9952376b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:03.824999 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a7e98-6e0e-4421-9b5b-0ba9952376b8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:03.846705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1213/2438] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:41:03 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:03.851023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a260de-d3f1-4670-ae07-83bd47df33cf admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:03.861659 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a260de-d3f1-4670-ae07-83bd47df33cf admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:03.867453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d239228-c5df-4cfe-8335-31355cb05562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:03.868121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d239228-c5df-4cfe-8335-31355cb05562 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.868404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d239228-c5df-4cfe-8335-31355cb05562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.868678 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d239228-c5df-4cfe-8335-31355cb05562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:03.869490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d239228-c5df-4cfe-8335-31355cb05562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:03.871112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d239228-c5df-4cfe-8335-31355cb05562 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:03.882193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1226/2439] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:03.886868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a7e98-6e0e-4421-9b5b-0ba9952376b8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:03.899859 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a7e98-6e0e-4421-9b5b-0ba9952376b8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:03.905797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c1134-b990-4956-b597-b9de2fec6001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:03.906463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c1134-b990-4956-b597-b9de2fec6001 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.906750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c1134-b990-4956-b597-b9de2fec6001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.907038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c1134-b990-4956-b597-b9de2fec6001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:03.907782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c1134-b990-4956-b597-b9de2fec6001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:03.909550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c1134-b990-4956-b597-b9de2fec6001 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:03.920566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1214/2440] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:03.930351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d239228-c5df-4cfe-8335-31355cb05562 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:03.941125 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d239228-c5df-4cfe-8335-31355cb05562 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:03.945998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7831f2-8a01-49d8-93ab-2b76b5f88b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:03.946803 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7831f2-8a01-49d8-93ab-2b76b5f88b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.947084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7831f2-8a01-49d8-93ab-2b76b5f88b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.947355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7831f2-8a01-49d8-93ab-2b76b5f88b7b admin admin] PATH_INFO: `/v3/projects/881cebad04fb4127b21ecda77dc26b31/users/865d1678cdf748728e1772bbc6600fa0/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:03.962585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7831f2-8a01-49d8-93ab-2b76b5f88b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=881cebad04fb4127b21ecda77dc26b31, user_id=865d1678cdf748728e1772bbc6600fa0, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:03.965280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7831f2-8a01-49d8-93ab-2b76b5f88b7b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:03.980833 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1227/2441] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:03 2022] PUT /identity/v3/projects/881cebad04fb4127b21ecda77dc26b31/users/865d1678cdf748728e1772bbc6600fa0/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:03.991194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e62a7e-c3bc-4601-9cfe-747ad76c9e4f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:03.991477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e62a7e-c3bc-4601-9cfe-747ad76c9e4f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:03.991754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e62a7e-c3bc-4601-9cfe-747ad76c9e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:04.018148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d3e62a7e-c3bc-4601-9cfe-747ad76c9e4f None None] MFA Rules not processed for user `865d1678cdf748728e1772bbc6600fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:04.019403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d3e62a7e-c3bc-4601-9cfe-747ad76c9e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:04.090583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1215/2442] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:04.095557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7831f2-8a01-49d8-93ab-2b76b5f88b7b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:04.158106 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7831f2-8a01-49d8-93ab-2b76b5f88b7b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:04.165519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0fb49-4f4c-42d1-ac60-66fbc24cb954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:04.166505 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0fb49-4f4c-42d1-ac60-66fbc24cb954 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:04.166865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0fb49-4f4c-42d1-ac60-66fbc24cb954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:04.167254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0fb49-4f4c-42d1-ac60-66fbc24cb954 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:04.168350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0fb49-4f4c-42d1-ac60-66fbc24cb954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:04.171503 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0fb49-4f4c-42d1-ac60-66fbc24cb954 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:04.205502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1228/2443] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:41:04 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:41:04.209940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e62a7e-c3bc-4601-9cfe-747ad76c9e4f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:04.221436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e62a7e-c3bc-4601-9cfe-747ad76c9e4f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:04.276674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e8986-10c4-4beb-a889-ee24d82fe900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:04.277376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e8986-10c4-4beb-a889-ee24d82fe900 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:04.277699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e8986-10c4-4beb-a889-ee24d82fe900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:04.277973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e8986-10c4-4beb-a889-ee24d82fe900 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:04.278674 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e8986-10c4-4beb-a889-ee24d82fe900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:04.280605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e8986-10c4-4beb-a889-ee24d82fe900 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:04.306614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1216/2444] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:41:04 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:04.312657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0fb49-4f4c-42d1-ac60-66fbc24cb954 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:04.323975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0fb49-4f4c-42d1-ac60-66fbc24cb954 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:04.329311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405b6ab-1d36-4198-bd62-d60ad5ef5c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:04.330009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405b6ab-1d36-4198-bd62-d60ad5ef5c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:04.330296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405b6ab-1d36-4198-bd62-d60ad5ef5c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:04.330574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405b6ab-1d36-4198-bd62-d60ad5ef5c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:04.331331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405b6ab-1d36-4198-bd62-d60ad5ef5c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:04.332987 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405b6ab-1d36-4198-bd62-d60ad5ef5c57 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:04.344857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1229/2445] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:04.349103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e8986-10c4-4beb-a889-ee24d82fe900 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:04.359975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e8986-10c4-4beb-a889-ee24d82fe900 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:04.365704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ec7bd-8fde-43fd-aaec-6afdc59d3a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:04.366600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ec7bd-8fde-43fd-aaec-6afdc59d3a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:04.366884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ec7bd-8fde-43fd-aaec-6afdc59d3a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:04.367159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ec7bd-8fde-43fd-aaec-6afdc59d3a5d admin admin] PATH_INFO: `/v3/projects/8ce4c32989bc446cab32801007bdc68f/users/38de8780014b41f8b7739d24fa0b470f/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:04.384058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ec7bd-8fde-43fd-aaec-6afdc59d3a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce4c32989bc446cab32801007bdc68f, user_id=38de8780014b41f8b7739d24fa0b470f, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:04.386736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ec7bd-8fde-43fd-aaec-6afdc59d3a5d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:04.402217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1217/2446] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:04 2022] PUT /identity/v3/projects/8ce4c32989bc446cab32801007bdc68f/users/38de8780014b41f8b7739d24fa0b470f/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:04.406296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405b6ab-1d36-4198-bd62-d60ad5ef5c57 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:04.416989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405b6ab-1d36-4198-bd62-d60ad5ef5c57 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:04.437212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:04.478009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee8739-c945-4f53-b4db-79ff9dfd4382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:04.478691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee8739-c945-4f53-b4db-79ff9dfd4382 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:04.478976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee8739-c945-4f53-b4db-79ff9dfd4382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:04.479252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee8739-c945-4f53-b4db-79ff9dfd4382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:04.479983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee8739-c945-4f53-b4db-79ff9dfd4382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:04.481638 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee8739-c945-4f53-b4db-79ff9dfd4382 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:04.491490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1230/2447] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:04.495940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ec7bd-8fde-43fd-aaec-6afdc59d3a5d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:04.501494 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:04.507493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ec7bd-8fde-43fd-aaec-6afdc59d3a5d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:04.513620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f93b1-6059-46cc-bdf1-9a6d558563ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:04.514514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f93b1-6059-46cc-bdf1-9a6d558563ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:04.514794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f93b1-6059-46cc-bdf1-9a6d558563ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:04.515047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f93b1-6059-46cc-bdf1-9a6d558563ce admin admin] PATH_INFO: `/v3/projects/8ce4c32989bc446cab32801007bdc68f/users/38de8780014b41f8b7739d24fa0b470f/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:04.517128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f93b1-6059-46cc-bdf1-9a6d558563ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce4c32989bc446cab32801007bdc68f, user_id=38de8780014b41f8b7739d24fa0b470f, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:04.519760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f93b1-6059-46cc-bdf1-9a6d558563ce admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:04.530797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1218/2448] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:04 2022] PUT /identity/v3/projects/8ce4c32989bc446cab32801007bdc68f/users/38de8780014b41f8b7739d24fa0b470f/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:04.541959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dfd69-b857-4d1e-809e-597dcb090466 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:04.542276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dfd69-b857-4d1e-809e-597dcb090466 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:04.542557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dfd69-b857-4d1e-809e-597dcb090466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:04.569968 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-964dfd69-b857-4d1e-809e-597dcb090466 None None] MFA Rules not processed for user `38de8780014b41f8b7739d24fa0b470f`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:04.571183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-964dfd69-b857-4d1e-809e-597dcb090466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:04.649041 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1231/2449] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:04.662345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f93b1-6059-46cc-bdf1-9a6d558563ce admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:04.664411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ab1f93b1-6059-46cc-bdf1-9a6d558563ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:04.732678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f93b1-6059-46cc-bdf1-9a6d558563ce admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:04.739026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece02d04-a42b-4802-8c45-9e6c51c2aa87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:04.739678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece02d04-a42b-4802-8c45-9e6c51c2aa87 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:04.739957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece02d04-a42b-4802-8c45-9e6c51c2aa87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:04.740232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece02d04-a42b-4802-8c45-9e6c51c2aa87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:04.797972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece02d04-a42b-4802-8c45-9e6c51c2aa87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:04.799395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece02d04-a42b-4802-8c45-9e6c51c2aa87 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:04.809972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1219/2450] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:05.402896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dfd69-b857-4d1e-809e-597dcb090466 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:05.404540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-964dfd69-b857-4d1e-809e-597dcb090466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:05.488730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dfd69-b857-4d1e-809e-597dcb090466 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:05.495668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393c390-1891-402c-950c-032a730bfea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:05.496340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393c390-1891-402c-950c-032a730bfea4 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:05.496630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393c390-1891-402c-950c-032a730bfea4 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:05.496945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393c390-1891-402c-950c-032a730bfea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:05.498868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0393c390-1891-402c-950c-032a730bfea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:05.543670 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:05.560840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393c390-1891-402c-950c-032a730bfea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:05.562456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393c390-1891-402c-950c-032a730bfea4 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:05.572040 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1232/2451] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:09.459884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece02d04-a42b-4802-8c45-9e6c51c2aa87 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:09.461524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ece02d04-a42b-4802-8c45-9e6c51c2aa87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:09.474124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece02d04-a42b-4802-8c45-9e6c51c2aa87 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:09.478381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60c942-1695-4f09-bd4d-d923257e3201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:09.479042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60c942-1695-4f09-bd4d-d923257e3201 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:09.479317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60c942-1695-4f09-bd4d-d923257e3201 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:09.479590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60c942-1695-4f09-bd4d-d923257e3201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:09.490514 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60c942-1695-4f09-bd4d-d923257e3201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:09.491946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60c942-1695-4f09-bd4d-d923257e3201 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:09.501827 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1220/2452] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:15.568456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:19.497384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=58,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:29.084228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3e8f2-3053-4e57-b49c-c659c2731937 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:29.084760 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3e8f2-3053-4e57-b49c-c659c2731937 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:29.084855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3e8f2-3053-4e57-b49c-c659c2731937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:29.103532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-83a3e8f2-3053-4e57-b49c-c659c2731937 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:29.104728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-83a3e8f2-3053-4e57-b49c-c659c2731937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:29.162824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1233/2453] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:41:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:29.213790 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60c942-1695-4f09-bd4d-d923257e3201 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:29.215247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3b60c942-1695-4f09-bd4d-d923257e3201 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:29.288949 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60c942-1695-4f09-bd4d-d923257e3201 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:29.294749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763eca5-a75d-4a5a-afc9-266a58dc3df5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:29.295387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763eca5-a75d-4a5a-afc9-266a58dc3df5 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:29.295668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763eca5-a75d-4a5a-afc9-266a58dc3df5 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:29.295941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763eca5-a75d-4a5a-afc9-266a58dc3df5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:29.304713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763eca5-a75d-4a5a-afc9-266a58dc3df5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:29.306327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763eca5-a75d-4a5a-afc9-266a58dc3df5 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:29.314689 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1221/2454] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:41:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:41:30.512198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3e8f2-3053-4e57-b49c-c659c2731937 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:30.599123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3e8f2-3053-4e57-b49c-c659c2731937 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:30.610850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:30.612252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:30.612540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:30.612859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] PATH_INFO: `/v3/users/865d1678cdf748728e1772bbc6600fa0` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:30.614728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=865d1678cdf748728e1772bbc6600fa0)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:30.617235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:30.689384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '865d1678cdf748728e1772bbc6600fa0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:30.706214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '865d1678cdf748728e1772bbc6600fa0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:30.722082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '865d1678cdf748728e1772bbc6600fa0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:30.722934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '865d1678cdf748728e1772bbc6600fa0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:30.737720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-855f07d3-a4af-4e12-ab22-300be73d35b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '865d1678cdf748728e1772bbc6600fa0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:30.739683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763eca5-a75d-4a5a-afc9-266a58dc3df5 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:30.751382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1234/2455] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:30 2022] DELETE /identity/v3/users/865d1678cdf748728e1772bbc6600fa0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:30.755276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce56fb7-b04f-403d-b5d3-9c29e6439060 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:30.755757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce56fb7-b04f-403d-b5d3-9c29e6439060 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:30.756185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce56fb7-b04f-403d-b5d3-9c29e6439060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:30.758683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763eca5-a75d-4a5a-afc9-266a58dc3df5 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:30.781628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7ce56fb7-b04f-403d-b5d3-9c29e6439060 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:30.783149 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7ce56fb7-b04f-403d-b5d3-9c29e6439060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:30.816108 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:30.868886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:30.870135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:30.870421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:30.870702 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:30.873380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:30.883161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1235/2456] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:30.891973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:30.893800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:30.894971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce56fb7-b04f-403d-b5d3-9c29e6439060 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:30.896491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7ce56fb7-b04f-403d-b5d3-9c29e6439060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:30.907465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1222/2457] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:30.986076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce56fb7-b04f-403d-b5d3-9c29e6439060 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:30.991894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ddf0c-3e4d-466b-ab17-f1f0242a3adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:30.992594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ddf0c-3e4d-466b-ab17-f1f0242a3adf service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:30.992993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ddf0c-3e4d-466b-ab17-f1f0242a3adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:30.993271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ddf0c-3e4d-466b-ab17-f1f0242a3adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:31.004059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ddf0c-3e4d-466b-ab17-f1f0242a3adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:31.005549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ddf0c-3e4d-466b-ab17-f1f0242a3adf service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:31.017473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1236/2458] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:31.326449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:31.332594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:31.378119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e9f32-2293-4f8a-a8f1-ee3850b606e7 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:31.383775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:31.385021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:31.385517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:31.385912 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] PATH_INFO: `/v3/projects/881cebad04fb4127b21ecda77dc26b31` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:31.387305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=881cebad04fb4127b21ecda77dc26b31)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:31.389732 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:31.466848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881cebad04fb4127b21ecda77dc26b31', 'request_id': 'req-8148eb4b-2198-4f23-bf68-def006d76097'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.475119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881cebad04fb4127b21ecda77dc26b31', 'request_id': 'req-8148eb4b-2198-4f23-bf68-def006d76097'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.494446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881cebad04fb4127b21ecda77dc26b31', 'request_id': 'req-8148eb4b-2198-4f23-bf68-def006d76097'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.503963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:31.510289 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] The token cache is being invalidate because project 881cebad04fb4127b21ecda77dc26b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:31.510935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.514225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.519527 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1223/2459] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:31 2022] DELETE /identity/v3/projects/881cebad04fb4127b21ecda77dc26b31 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:31.523366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ddf0c-3e4d-466b-ab17-f1f0242a3adf service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:31.540454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ddf0c-3e4d-466b-ab17-f1f0242a3adf service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:31.621836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:31.623110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:31.623500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:31.623828 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] PATH_INFO: `/v3/users/38de8780014b41f8b7739d24fa0b470f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:31.625252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38de8780014b41f8b7739d24fa0b470f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:31.627635 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:31.707387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38de8780014b41f8b7739d24fa0b470f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.725121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38de8780014b41f8b7739d24fa0b470f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.731527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:31.738238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38de8780014b41f8b7739d24fa0b470f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.739637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38de8780014b41f8b7739d24fa0b470f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.764846 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6e0dd50d-0038-4f05-be5c-0ae1b0663352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38de8780014b41f8b7739d24fa0b470f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:31.783356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1237/2460] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:31 2022] DELETE /identity/v3/users/38de8780014b41f8b7739d24fa0b470f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:32.148934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:32.150261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:32.168335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148eb4b-2198-4f23-bf68-def006d76097 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:32.247271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:32.248499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.248979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.249253 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] PATH_INFO: `/v3/projects/8ce4c32989bc446cab32801007bdc68f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.250574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ce4c32989bc446cab32801007bdc68f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:32.252583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:32.316814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ce4c32989bc446cab32801007bdc68f', 'request_id': 'req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:32.324083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ce4c32989bc446cab32801007bdc68f', 'request_id': 'req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:32.334061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ce4c32989bc446cab32801007bdc68f', 'request_id': 'req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:32.345643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] The token cache is being invalidate because project 8ce4c32989bc446cab32801007bdc68f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:32.345982 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:32.346606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8c3a4693-451a-4c08-a8b2-e23e0f5e0a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:32.353215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2c872-0a08-4131-84b7-db7098bddb32 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.356788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2c872-0a08-4131-84b7-db7098bddb32 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.356788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2c872-0a08-4131-84b7-db7098bddb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.357043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1224/2461] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:32 2022] DELETE /identity/v3/projects/8ce4c32989bc446cab32801007bdc68f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:32.395872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-03d2c872-0a08-4131-84b7-db7098bddb32 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:32.397386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-03d2c872-0a08-4131-84b7-db7098bddb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:32.469594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1238/2462] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:32.478314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45e7ec-bd58-4bbd-98d2-29ae0dec0cda None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.478826 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45e7ec-bd58-4bbd-98d2-29ae0dec0cda None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.479213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45e7ec-bd58-4bbd-98d2-29ae0dec0cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.499294 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1e45e7ec-bd58-4bbd-98d2-29ae0dec0cda None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:32.500864 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1e45e7ec-bd58-4bbd-98d2-29ae0dec0cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:32.508942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1225/2463] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:32.513374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2c872-0a08-4131-84b7-db7098bddb32 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:32.523663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2c872-0a08-4131-84b7-db7098bddb32 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:32.528808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4f41f-433c-45ae-a835-ce85902c9e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:32.529584 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4f41f-433c-45ae-a835-ce85902c9e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.529975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4f41f-433c-45ae-a835-ce85902c9e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.530349 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4f41f-433c-45ae-a835-ce85902c9e96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.531198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4f41f-433c-45ae-a835-ce85902c9e96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:41:32.531607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4f41f-433c-45ae-a835-ce85902c9e96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:32.533404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4f41f-433c-45ae-a835-ce85902c9e96 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:32.546676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1239/2464] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:41:32.551618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45e7ec-bd58-4bbd-98d2-29ae0dec0cda None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:32.561339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45e7ec-bd58-4bbd-98d2-29ae0dec0cda None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:32.567071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef787214-bac8-49d8-900c-fc124f99ddec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:32.567807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef787214-bac8-49d8-900c-fc124f99ddec admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.568184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef787214-bac8-49d8-900c-fc124f99ddec admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.568562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef787214-bac8-49d8-900c-fc124f99ddec admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.569397 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef787214-bac8-49d8-900c-fc124f99ddec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:32.571442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef787214-bac8-49d8-900c-fc124f99ddec admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:32.605306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1226/2465] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:41:32 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:32.606881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4f41f-433c-45ae-a835-ce85902c9e96 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:32.617797 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4f41f-433c-45ae-a835-ce85902c9e96 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:32.690930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211ad53-a77e-4ad8-b053-bb2b25f37cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:32.691947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211ad53-a77e-4ad8-b053-bb2b25f37cde admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.692411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211ad53-a77e-4ad8-b053-bb2b25f37cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.692699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211ad53-a77e-4ad8-b053-bb2b25f37cde admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.693438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211ad53-a77e-4ad8-b053-bb2b25f37cde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:32.695415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211ad53-a77e-4ad8-b053-bb2b25f37cde admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:32.720368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1240/2466] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:41:32 2022] POST /identity/v3/users => generated 301 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:41:32.724484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef787214-bac8-49d8-900c-fc124f99ddec admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:32.736178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef787214-bac8-49d8-900c-fc124f99ddec admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:32.746107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34bfe8-b6c1-4afc-902e-df2543d69b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:32.747075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34bfe8-b6c1-4afc-902e-df2543d69b21 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.747542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34bfe8-b6c1-4afc-902e-df2543d69b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.747830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34bfe8-b6c1-4afc-902e-df2543d69b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.748584 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34bfe8-b6c1-4afc-902e-df2543d69b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:32.750260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34bfe8-b6c1-4afc-902e-df2543d69b21 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:32.772541 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1227/2467] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:32.773829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211ad53-a77e-4ad8-b053-bb2b25f37cde admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:32.785857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211ad53-a77e-4ad8-b053-bb2b25f37cde admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:32.791663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a0c75-edee-4a55-9cbe-822860bbd292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:32.792671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a0c75-edee-4a55-9cbe-822860bbd292 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.793251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a0c75-edee-4a55-9cbe-822860bbd292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.793849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a0c75-edee-4a55-9cbe-822860bbd292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.794950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a0c75-edee-4a55-9cbe-822860bbd292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:32.796908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a0c75-edee-4a55-9cbe-822860bbd292 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:32.812286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1241/2468] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:32.813586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34bfe8-b6c1-4afc-902e-df2543d69b21 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:32.829248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34bfe8-b6c1-4afc-902e-df2543d69b21 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:32.835345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbe4d7-ed56-41bb-a5b9-b1b6dcd691b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:32.836330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbe4d7-ed56-41bb-a5b9-b1b6dcd691b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.836685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbe4d7-ed56-41bb-a5b9-b1b6dcd691b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.837069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbe4d7-ed56-41bb-a5b9-b1b6dcd691b9 admin admin] PATH_INFO: `/v3/projects/911dfc5fa9584f9eaa89700c78f56ab6/users/51e72eb2e74a414f849cf0768a7c38ad/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.864558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbe4d7-ed56-41bb-a5b9-b1b6dcd691b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=911dfc5fa9584f9eaa89700c78f56ab6, user_id=51e72eb2e74a414f849cf0768a7c38ad, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:32.867687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbe4d7-ed56-41bb-a5b9-b1b6dcd691b9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:32.900137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1228/2469] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:32 2022] PUT /identity/v3/projects/911dfc5fa9584f9eaa89700c78f56ab6/users/51e72eb2e74a414f849cf0768a7c38ad/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:32.903841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565fd84-fdca-4a48-98bc-4009abfda68e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:32.904392 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565fd84-fdca-4a48-98bc-4009abfda68e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:32.904855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565fd84-fdca-4a48-98bc-4009abfda68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:32.941322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6565fd84-fdca-4a48-98bc-4009abfda68e None None] MFA Rules not processed for user `51e72eb2e74a414f849cf0768a7c38ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:32.942807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6565fd84-fdca-4a48-98bc-4009abfda68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:33.003395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:33.041265 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1242/2470] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:33.042278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbe4d7-ed56-41bb-a5b9-b1b6dcd691b9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:33.113372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:33.196444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbe4d7-ed56-41bb-a5b9-b1b6dcd691b9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:33.220016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9728c-c3d6-482a-967b-51fd227fcc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:33.223130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9728c-c3d6-482a-967b-51fd227fcc59 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:33.223249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9728c-c3d6-482a-967b-51fd227fcc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:33.223656 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9728c-c3d6-482a-967b-51fd227fcc59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:33.225542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9728c-c3d6-482a-967b-51fd227fcc59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:33.231821 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9728c-c3d6-482a-967b-51fd227fcc59 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:33.275967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1229/2471] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:41:33 2022] POST /identity/v3/projects => generated 401 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:41:33.282359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565fd84-fdca-4a48-98bc-4009abfda68e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:33.302486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565fd84-fdca-4a48-98bc-4009abfda68e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:33.413893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01609d-a61f-41de-8e4c-4eb0f5e4c8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:33.414692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01609d-a61f-41de-8e4c-4eb0f5e4c8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:33.415043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01609d-a61f-41de-8e4c-4eb0f5e4c8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:33.415386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01609d-a61f-41de-8e4c-4eb0f5e4c8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:33.416155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01609d-a61f-41de-8e4c-4eb0f5e4c8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:33.418943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01609d-a61f-41de-8e4c-4eb0f5e4c8b7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:33.453190 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1243/2472] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:41:33 2022] POST /identity/v3/users => generated 307 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:33.457298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9728c-c3d6-482a-967b-51fd227fcc59 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:33.472687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9728c-c3d6-482a-967b-51fd227fcc59 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:33.478444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bba5e8-af41-4790-8b00-13c42ac65fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:33.479247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bba5e8-af41-4790-8b00-13c42ac65fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:33.479625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bba5e8-af41-4790-8b00-13c42ac65fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:33.479997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bba5e8-af41-4790-8b00-13c42ac65fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:33.481459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bba5e8-af41-4790-8b00-13c42ac65fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:33.484183 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bba5e8-af41-4790-8b00-13c42ac65fc3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:33.502626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1230/2473] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:33.503891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01609d-a61f-41de-8e4c-4eb0f5e4c8b7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:33.514919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01609d-a61f-41de-8e4c-4eb0f5e4c8b7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:33.522082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae68a8-a7c7-4422-b625-be7bbafd8d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:33.522991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae68a8-a7c7-4422-b625-be7bbafd8d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:33.523275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae68a8-a7c7-4422-b625-be7bbafd8d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:33.523552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae68a8-a7c7-4422-b625-be7bbafd8d1b admin admin] PATH_INFO: `/v3/projects/ede0d941e4dc48bcae40ba20e86afa26/users/184dcd305a334d458732b04ab1944060/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:33.544937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae68a8-a7c7-4422-b625-be7bbafd8d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede0d941e4dc48bcae40ba20e86afa26, user_id=184dcd305a334d458732b04ab1944060, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:33.548531 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae68a8-a7c7-4422-b625-be7bbafd8d1b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:33.570084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1244/2474] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:33 2022] PUT /identity/v3/projects/ede0d941e4dc48bcae40ba20e86afa26/users/184dcd305a334d458732b04ab1944060/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:33.575770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bba5e8-af41-4790-8b00-13c42ac65fc3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:33.588317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bba5e8-af41-4790-8b00-13c42ac65fc3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:33.696871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9891ae-8ec6-4185-8f22-7a7c1e30402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:33.697710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9891ae-8ec6-4185-8f22-7a7c1e30402e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:33.698004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9891ae-8ec6-4185-8f22-7a7c1e30402e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:33.698285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9891ae-8ec6-4185-8f22-7a7c1e30402e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:33.699742 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9891ae-8ec6-4185-8f22-7a7c1e30402e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:33.702116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9891ae-8ec6-4185-8f22-7a7c1e30402e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:33.716389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1231/2475] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:33.721416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae68a8-a7c7-4422-b625-be7bbafd8d1b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:33.736782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae68a8-a7c7-4422-b625-be7bbafd8d1b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:33.743313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4385309-df0b-448b-bb42-dca2370be0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:33.745434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4385309-df0b-448b-bb42-dca2370be0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:33.745693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4385309-df0b-448b-bb42-dca2370be0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:33.745965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4385309-df0b-448b-bb42-dca2370be0eb admin admin] PATH_INFO: `/v3/projects/ede0d941e4dc48bcae40ba20e86afa26/users/184dcd305a334d458732b04ab1944060/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:33.753367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4385309-df0b-448b-bb42-dca2370be0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede0d941e4dc48bcae40ba20e86afa26, user_id=184dcd305a334d458732b04ab1944060, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:33.756724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4385309-df0b-448b-bb42-dca2370be0eb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:33.771827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1245/2476] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:33 2022] PUT /identity/v3/projects/ede0d941e4dc48bcae40ba20e86afa26/users/184dcd305a334d458732b04ab1944060/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:41:33.784115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1f649-37c4-4b15-bb74-f94e80f8bb53 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:33.784404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1f649-37c4-4b15-bb74-f94e80f8bb53 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:33.784684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1f649-37c4-4b15-bb74-f94e80f8bb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:33.819172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-44c1f649-37c4-4b15-bb74-f94e80f8bb53 None None] MFA Rules not processed for user `184dcd305a334d458732b04ab1944060`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:33.820706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-44c1f649-37c4-4b15-bb74-f94e80f8bb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:33.919110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1232/2477] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:33.930986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4385309-df0b-448b-bb42-dca2370be0eb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:33.932955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d4385309-df0b-448b-bb42-dca2370be0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:34.005842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4385309-df0b-448b-bb42-dca2370be0eb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:34.010623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7771c9-2303-4c83-9478-7d1530e383cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:34.011389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7771c9-2303-4c83-9478-7d1530e383cf service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:34.012024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7771c9-2303-4c83-9478-7d1530e383cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:34.012624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7771c9-2303-4c83-9478-7d1530e383cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:34.042321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7771c9-2303-4c83-9478-7d1530e383cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:34.044235 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7771c9-2303-4c83-9478-7d1530e383cf service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:34.055773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1246/2478] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:34.145089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1f649-37c4-4b15-bb74-f94e80f8bb53 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:34.158568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1f649-37c4-4b15-bb74-f94e80f8bb53 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:34.162804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e4712-40e7-49b0-86e3-111780b35100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:34.163600 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e4712-40e7-49b0-86e3-111780b35100 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:34.163985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e4712-40e7-49b0-86e3-111780b35100 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:34.164363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e4712-40e7-49b0-86e3-111780b35100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:34.184545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:34.243920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e4712-40e7-49b0-86e3-111780b35100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:34.247045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e4712-40e7-49b0-86e3-111780b35100 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:34.260174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1233/2479] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:36.325457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7771c9-2303-4c83-9478-7d1530e383cf service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:36.326667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2c7771c9-2303-4c83-9478-7d1530e383cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:36.381798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:36.401187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7771c9-2303-4c83-9478-7d1530e383cf service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:36.407377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841f501-799e-41b5-a85f-09613b081eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:36.408509 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841f501-799e-41b5-a85f-09613b081eac service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:36.408791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841f501-799e-41b5-a85f-09613b081eac service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:36.409081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841f501-799e-41b5-a85f-09613b081eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:36.411000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f841f501-799e-41b5-a85f-09613b081eac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:36.422078 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841f501-799e-41b5-a85f-09613b081eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:36.423554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841f501-799e-41b5-a85f-09613b081eac service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:36.433711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1247/2480] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:38.618892 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e4712-40e7-49b0-86e3-111780b35100 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:38.700337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e4712-40e7-49b0-86e3-111780b35100 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:38.706401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:38.707894 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:38.708190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:38.708465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] PATH_INFO: `/v3/users/51e72eb2e74a414f849cf0768a7c38ad` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:38.709789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e72eb2e74a414f849cf0768a7c38ad)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:38.712044 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:38.772141 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e72eb2e74a414f849cf0768a7c38ad'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:38.785608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e72eb2e74a414f849cf0768a7c38ad'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:38.794785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e72eb2e74a414f849cf0768a7c38ad'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:38.802797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e72eb2e74a414f849cf0768a7c38ad'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:38.803413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e72eb2e74a414f849cf0768a7c38ad'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:38.810566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1234/2481] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:38 2022] DELETE /identity/v3/users/51e72eb2e74a414f849cf0768a7c38ad => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:38.815169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177a190-4791-4e72-9b32-17a211ae89ec None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:38.815640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177a190-4791-4e72-9b32-17a211ae89ec None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:38.815731 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177a190-4791-4e72-9b32-17a211ae89ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:38.833579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2177a190-4791-4e72-9b32-17a211ae89ec None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:38.834753 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2177a190-4791-4e72-9b32-17a211ae89ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:38.900857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1248/2482] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:38.911287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:38.913075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:38.958729 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:38.976484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4bd1f-bc77-4ee5-b26d-6a7428a01c59 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:38.980103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185487c-4b4f-45f2-8960-f7cf653c6bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:38.981061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185487c-4b4f-45f2-8960-f7cf653c6bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:38.981443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185487c-4b4f-45f2-8960-f7cf653c6bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:38.981670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185487c-4b4f-45f2-8960-f7cf653c6bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:38.991057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185487c-4b4f-45f2-8960-f7cf653c6bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:38.992497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185487c-4b4f-45f2-8960-f7cf653c6bd3 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:39.002543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1235/2483] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:39.282147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177a190-4791-4e72-9b32-17a211ae89ec None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:39.283908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2177a190-4791-4e72-9b32-17a211ae89ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:39.302615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177a190-4791-4e72-9b32-17a211ae89ec None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:39.307789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:39.309113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:39.309113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:39.309442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] PATH_INFO: `/v3/projects/911dfc5fa9584f9eaa89700c78f56ab6` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:39.310608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] RBAC: Authorizing `identity:delete_project(project_id=911dfc5fa9584f9eaa89700c78f56ab6)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:39.312587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:39.361895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '911dfc5fa9584f9eaa89700c78f56ab6', 'request_id': 'req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.370691 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '911dfc5fa9584f9eaa89700c78f56ab6', 'request_id': 'req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.377571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '911dfc5fa9584f9eaa89700c78f56ab6', 'request_id': 'req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.390550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] The token cache is being invalidate because project 911dfc5fa9584f9eaa89700c78f56ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:39.390871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.391648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.393575 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1249/2484] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:39 2022] DELETE /identity/v3/projects/911dfc5fa9584f9eaa89700c78f56ab6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:39.397443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185487c-4b4f-45f2-8960-f7cf653c6bd3 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:39.407492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185487c-4b4f-45f2-8960-f7cf653c6bd3 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:39.481445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:39.482332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:39.482612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:39.482997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] PATH_INFO: `/v3/users/184dcd305a334d458732b04ab1944060` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:39.484175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=184dcd305a334d458732b04ab1944060)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:39.486142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:39.534175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184dcd305a334d458732b04ab1944060'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.546636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184dcd305a334d458732b04ab1944060'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.556491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184dcd305a334d458732b04ab1944060'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.564307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184dcd305a334d458732b04ab1944060'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.564991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b35a5204-8447-4b5c-ad4a-4b0a7377e9d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184dcd305a334d458732b04ab1944060'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:39.570886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1236/2485] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:39 2022] DELETE /identity/v3/users/184dcd305a334d458732b04ab1944060 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:39.918654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:39.920284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:39.936993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c13f9e-96cb-4bb5-bfa7-3e1b068f414e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:39.998166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:39.998940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:39.999222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:39.999501 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] PATH_INFO: `/v3/projects/ede0d941e4dc48bcae40ba20e86afa26` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.000707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede0d941e4dc48bcae40ba20e86afa26)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.002662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.008992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:40.054267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede0d941e4dc48bcae40ba20e86afa26', 'request_id': 'req-1ad3b03b-31d6-487b-a08b-5fd718bcb663'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:40.063739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede0d941e4dc48bcae40ba20e86afa26', 'request_id': 'req-1ad3b03b-31d6-487b-a08b-5fd718bcb663'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:40.071144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede0d941e4dc48bcae40ba20e86afa26', 'request_id': 'req-1ad3b03b-31d6-487b-a08b-5fd718bcb663'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:40.085330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] The token cache is being invalidate because project ede0d941e4dc48bcae40ba20e86afa26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:40.085630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:40.086593 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1ad3b03b-31d6-487b-a08b-5fd718bcb663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:40.088264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1250/2486] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:39 2022] DELETE /identity/v3/projects/ede0d941e4dc48bcae40ba20e86afa26 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:40.093768 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11036e13-79d3-4bfe-84ad-2934ee4d1b92 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.094629 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11036e13-79d3-4bfe-84ad-2934ee4d1b92 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.094789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11036e13-79d3-4bfe-84ad-2934ee4d1b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.115314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-11036e13-79d3-4bfe-84ad-2934ee4d1b92 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:40.116536 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-11036e13-79d3-4bfe-84ad-2934ee4d1b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:40.174859 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:40.188895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1237/2487] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:40.193855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5db91-bf4a-4898-a62e-bf32882e0b93 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.194277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5db91-bf4a-4898-a62e-bf32882e0b93 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.195432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5db91-bf4a-4898-a62e-bf32882e0b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.218484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-88f5db91-bf4a-4898-a62e-bf32882e0b93 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:40.220183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-88f5db91-bf4a-4898-a62e-bf32882e0b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:40.229470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1251/2488] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:40.233888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11036e13-79d3-4bfe-84ad-2934ee4d1b92 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.244150 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11036e13-79d3-4bfe-84ad-2934ee4d1b92 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.248507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4ef26-57a2-4535-8921-68a22f2433d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.249195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4ef26-57a2-4535-8921-68a22f2433d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.249490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4ef26-57a2-4535-8921-68a22f2433d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.249838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4ef26-57a2-4535-8921-68a22f2433d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.250585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4ef26-57a2-4535-8921-68a22f2433d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:41:40.250886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4ef26-57a2-4535-8921-68a22f2433d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.252510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4ef26-57a2-4535-8921-68a22f2433d0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.265824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1238/2489] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:41:40.270348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5db91-bf4a-4898-a62e-bf32882e0b93 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.280033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5db91-bf4a-4898-a62e-bf32882e0b93 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.285057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0d3c5-5792-4f7a-a25b-451be5692414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.285714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0d3c5-5792-4f7a-a25b-451be5692414 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.285993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0d3c5-5792-4f7a-a25b-451be5692414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.286264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0d3c5-5792-4f7a-a25b-451be5692414 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.286949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0d3c5-5792-4f7a-a25b-451be5692414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.288890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0d3c5-5792-4f7a-a25b-451be5692414 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.319350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1252/2490] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:41:40 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:40.320486 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4ef26-57a2-4535-8921-68a22f2433d0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.330627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4ef26-57a2-4535-8921-68a22f2433d0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.390125 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58178cb6-5260-4292-bdbc-366bfa8076ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.390809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58178cb6-5260-4292-bdbc-366bfa8076ff admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.391093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58178cb6-5260-4292-bdbc-366bfa8076ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.391373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58178cb6-5260-4292-bdbc-366bfa8076ff admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.392069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58178cb6-5260-4292-bdbc-366bfa8076ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.394049 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58178cb6-5260-4292-bdbc-366bfa8076ff admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.418409 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1239/2491] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:41:40 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:40.422976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0d3c5-5792-4f7a-a25b-451be5692414 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.433660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0d3c5-5792-4f7a-a25b-451be5692414 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.439848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892430e-bbf4-40cf-90c4-5fd276cfd3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.440537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892430e-bbf4-40cf-90c4-5fd276cfd3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.440855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892430e-bbf4-40cf-90c4-5fd276cfd3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.441140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892430e-bbf4-40cf-90c4-5fd276cfd3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.441943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892430e-bbf4-40cf-90c4-5fd276cfd3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.443579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892430e-bbf4-40cf-90c4-5fd276cfd3a3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.459953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58178cb6-5260-4292-bdbc-366bfa8076ff admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.460634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1253/2492] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:40.471421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58178cb6-5260-4292-bdbc-366bfa8076ff admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.476668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cae0e-de00-44c3-86c5-a41ce153b68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.477376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cae0e-de00-44c3-86c5-a41ce153b68e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.477665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cae0e-de00-44c3-86c5-a41ce153b68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.477940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cae0e-de00-44c3-86c5-a41ce153b68e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.478687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cae0e-de00-44c3-86c5-a41ce153b68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.480516 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cae0e-de00-44c3-86c5-a41ce153b68e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.493313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1240/2493] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:41:40.498229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892430e-bbf4-40cf-90c4-5fd276cfd3a3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.508618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892430e-bbf4-40cf-90c4-5fd276cfd3a3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.517315 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4b833-992d-4d7c-a047-9f81f4fe3928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.518244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b833-992d-4d7c-a047-9f81f4fe3928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.518516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b833-992d-4d7c-a047-9f81f4fe3928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.518794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b833-992d-4d7c-a047-9f81f4fe3928 admin admin] PATH_INFO: `/v3/projects/02ac270c82754a5a96b0d9cb6ac2a199/users/9e0113fe857b48898382288b134f99e9/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.535202 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4b833-992d-4d7c-a047-9f81f4fe3928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ac270c82754a5a96b0d9cb6ac2a199, user_id=9e0113fe857b48898382288b134f99e9, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.537934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4b833-992d-4d7c-a047-9f81f4fe3928 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.553310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1254/2494] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:40 2022] PUT /identity/v3/projects/02ac270c82754a5a96b0d9cb6ac2a199/users/9e0113fe857b48898382288b134f99e9/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:40.565473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4a4e-aaf0-49cf-abf6-d2b9c236bdc2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.565763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4a4e-aaf0-49cf-abf6-d2b9c236bdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.566040 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4a4e-aaf0-49cf-abf6-d2b9c236bdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.595908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-46fa4a4e-aaf0-49cf-abf6-d2b9c236bdc2 None None] MFA Rules not processed for user `9e0113fe857b48898382288b134f99e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:40.597159 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-46fa4a4e-aaf0-49cf-abf6-d2b9c236bdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:40.662959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1241/2495] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:40.667684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4b833-992d-4d7c-a047-9f81f4fe3928 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.737255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4b833-992d-4d7c-a047-9f81f4fe3928 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.742083 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0f8ee-f25b-4c1c-bc5e-b8980a0f4cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.742739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0f8ee-f25b-4c1c-bc5e-b8980a0f4cde admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.743021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0f8ee-f25b-4c1c-bc5e-b8980a0f4cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.743268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0f8ee-f25b-4c1c-bc5e-b8980a0f4cde admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.743966 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0f8ee-f25b-4c1c-bc5e-b8980a0f4cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.745912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0f8ee-f25b-4c1c-bc5e-b8980a0f4cde admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.774734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1255/2496] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:41:40 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:41:40.779075 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4a4e-aaf0-49cf-abf6-d2b9c236bdc2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.790525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4a4e-aaf0-49cf-abf6-d2b9c236bdc2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.845641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:40.853644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797ddf2-69da-40d1-88af-833e83e88190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.854642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797ddf2-69da-40d1-88af-833e83e88190 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.855035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797ddf2-69da-40d1-88af-833e83e88190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.855444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797ddf2-69da-40d1-88af-833e83e88190 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.856160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797ddf2-69da-40d1-88af-833e83e88190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.858103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797ddf2-69da-40d1-88af-833e83e88190 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.881795 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1242/2497] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:41:40 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:40.885970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0f8ee-f25b-4c1c-bc5e-b8980a0f4cde admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.896896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0f8ee-f25b-4c1c-bc5e-b8980a0f4cde admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.901749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286bfef-4a08-4f9e-a916-738b1330ebf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.902420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286bfef-4a08-4f9e-a916-738b1330ebf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.902774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286bfef-4a08-4f9e-a916-738b1330ebf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.903051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286bfef-4a08-4f9e-a916-738b1330ebf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.903908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286bfef-4a08-4f9e-a916-738b1330ebf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.905549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286bfef-4a08-4f9e-a916-738b1330ebf9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.917715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1256/2498] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:40.922163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797ddf2-69da-40d1-88af-833e83e88190 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.931962 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797ddf2-69da-40d1-88af-833e83e88190 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:40.936889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f903ab-ec48-4b67-a03f-e132faf83112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:40.937774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f903ab-ec48-4b67-a03f-e132faf83112 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:40.938055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f903ab-ec48-4b67-a03f-e132faf83112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:40.938325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f903ab-ec48-4b67-a03f-e132faf83112 admin admin] PATH_INFO: `/v3/projects/4e57acafb4f14db2a10926c044d458db/users/380b6a47de1540d091e75a724b204be5/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:40.954326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f903ab-ec48-4b67-a03f-e132faf83112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e57acafb4f14db2a10926c044d458db, user_id=380b6a47de1540d091e75a724b204be5, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:40.957333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f903ab-ec48-4b67-a03f-e132faf83112 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:40.972891 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1243/2499] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:40 2022] PUT /identity/v3/projects/4e57acafb4f14db2a10926c044d458db/users/380b6a47de1540d091e75a724b204be5/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:40.977291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286bfef-4a08-4f9e-a916-738b1330ebf9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:40.983939 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:40.992428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286bfef-4a08-4f9e-a916-738b1330ebf9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:41.053051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379a01-1690-4154-90e0-68d8cd726b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:41.053789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379a01-1690-4154-90e0-68d8cd726b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:41.054073 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379a01-1690-4154-90e0-68d8cd726b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:41.054351 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379a01-1690-4154-90e0-68d8cd726b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:41.055088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8379a01-1690-4154-90e0-68d8cd726b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:41.056778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8379a01-1690-4154-90e0-68d8cd726b23 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:41.070526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1257/2500] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:41.072209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f903ab-ec48-4b67-a03f-e132faf83112 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:41.082507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f903ab-ec48-4b67-a03f-e132faf83112 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:41.087011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade2177-4b77-4923-82c8-4245bc8da469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:41.087892 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade2177-4b77-4923-82c8-4245bc8da469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:41.088167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade2177-4b77-4923-82c8-4245bc8da469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:41.088437 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade2177-4b77-4923-82c8-4245bc8da469 admin admin] PATH_INFO: `/v3/projects/4e57acafb4f14db2a10926c044d458db/users/380b6a47de1540d091e75a724b204be5/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:41.090506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade2177-4b77-4923-82c8-4245bc8da469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e57acafb4f14db2a10926c044d458db, user_id=380b6a47de1540d091e75a724b204be5, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:41.093181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade2177-4b77-4923-82c8-4245bc8da469 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:41.104033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1244/2501] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:41 2022] PUT /identity/v3/projects/4e57acafb4f14db2a10926c044d458db/users/380b6a47de1540d091e75a724b204be5/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:41.115112 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bae64-bd21-4666-98c1-49d21ec2d398 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:41.115495 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bae64-bd21-4666-98c1-49d21ec2d398 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:41.115808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bae64-bd21-4666-98c1-49d21ec2d398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:41.143545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-297bae64-bd21-4666-98c1-49d21ec2d398 None None] MFA Rules not processed for user `380b6a47de1540d091e75a724b204be5`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:41.144884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-297bae64-bd21-4666-98c1-49d21ec2d398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:41.219082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1258/2502] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:41.229483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade2177-4b77-4923-82c8-4245bc8da469 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:41.231490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2ade2177-4b77-4923-82c8-4245bc8da469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:41.296384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade2177-4b77-4923-82c8-4245bc8da469 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:41.300166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f31516-4113-4637-8c00-74c0c833a155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:41.300864 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f31516-4113-4637-8c00-74c0c833a155 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:41.301184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f31516-4113-4637-8c00-74c0c833a155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:41.301522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f31516-4113-4637-8c00-74c0c833a155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:41.311614 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f31516-4113-4637-8c00-74c0c833a155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:41.313098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f31516-4113-4637-8c00-74c0c833a155 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:41.323192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1245/2503] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:41.926686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bae64-bd21-4666-98c1-49d21ec2d398 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:41.991877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bae64-bd21-4666-98c1-49d21ec2d398 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:41.996637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:41.997664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:41.997950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:41.998229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] PATH_INFO: `/v3/users/9e0113fe857b48898382288b134f99e9` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:41.999557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0113fe857b48898382288b134f99e9)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:42.001577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:42.034053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:42.056134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0113fe857b48898382288b134f99e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.075019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0113fe857b48898382288b134f99e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.084363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0113fe857b48898382288b134f99e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.085130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0113fe857b48898382288b134f99e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.092384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0113fe857b48898382288b134f99e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.098727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1259/2504] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:41 2022] DELETE /identity/v3/users/9e0113fe857b48898382288b134f99e9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:42.104169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5950-21c1-4b0e-bdb2-8de51a51d2b1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:42.104169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5950-21c1-4b0e-bdb2-8de51a51d2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:42.104619 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5950-21c1-4b0e-bdb2-8de51a51d2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:42.123758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-43aa5950-21c1-4b0e-bdb2-8de51a51d2b1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:42.125141 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-43aa5950-21c1-4b0e-bdb2-8de51a51d2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:42.188310 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1246/2505] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:42.198943 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:42.200573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:42.263171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60ae87-b855-435d-8241-bb42c0b47fa4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:42.267144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6dc8e-f4b1-4484-8911-d90264501c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:42.267838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6dc8e-f4b1-4484-8911-d90264501c9c service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:42.268148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6dc8e-f4b1-4484-8911-d90264501c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:42.268407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6dc8e-f4b1-4484-8911-d90264501c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:42.277960 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6dc8e-f4b1-4484-8911-d90264501c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:42.279441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6dc8e-f4b1-4484-8911-d90264501c9c service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:42.289446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1260/2506] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:42.578444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5950-21c1-4b0e-bdb2-8de51a51d2b1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:42.579960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-43aa5950-21c1-4b0e-bdb2-8de51a51d2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:42.592565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5950-21c1-4b0e-bdb2-8de51a51d2b1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:42.599617 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:42.600853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:42.601217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:42.601551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] PATH_INFO: `/v3/projects/02ac270c82754a5a96b0d9cb6ac2a199` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:42.602990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ac270c82754a5a96b0d9cb6ac2a199)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:42.605061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:42.671460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:42.683719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ac270c82754a5a96b0d9cb6ac2a199', 'request_id': 'req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.689855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ac270c82754a5a96b0d9cb6ac2a199', 'request_id': 'req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.702064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ac270c82754a5a96b0d9cb6ac2a199', 'request_id': 'req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.715668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] The token cache is being invalidate because project 02ac270c82754a5a96b0d9cb6ac2a199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:42.716035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.717321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.718831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1247/2507] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:42 2022] DELETE /identity/v3/projects/02ac270c82754a5a96b0d9cb6ac2a199 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:42.723042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6dc8e-f4b1-4484-8911-d90264501c9c service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:42.733641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6dc8e-f4b1-4484-8911-d90264501c9c service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:42.818524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:42.819416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:42.819699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:42.819974 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] PATH_INFO: `/v3/users/380b6a47de1540d091e75a724b204be5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:42.821221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=380b6a47de1540d091e75a724b204be5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:42.823174 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:42.876041 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '380b6a47de1540d091e75a724b204be5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.878403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:42.892441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '380b6a47de1540d091e75a724b204be5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.901410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '380b6a47de1540d091e75a724b204be5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.902706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '380b6a47de1540d091e75a724b204be5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.910408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-460fe376-a54a-4842-98cd-57a703819e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '380b6a47de1540d091e75a724b204be5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:42.916958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1261/2508] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:42 2022] DELETE /identity/v3/users/380b6a47de1540d091e75a724b204be5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:43.239936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:43.241687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:43.255880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438dcdb-e9bd-4117-a4e7-8fe750b6e96a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:43.318171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:43.318993 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.319277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.319587 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] PATH_INFO: `/v3/projects/4e57acafb4f14db2a10926c044d458db` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.320756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e57acafb4f14db2a10926c044d458db)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:43.322675 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:43.373284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e57acafb4f14db2a10926c044d458db', 'request_id': 'req-59c46076-3007-4346-967a-0d3ccc120f0c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:43.380136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e57acafb4f14db2a10926c044d458db', 'request_id': 'req-59c46076-3007-4346-967a-0d3ccc120f0c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:43.388459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e57acafb4f14db2a10926c044d458db', 'request_id': 'req-59c46076-3007-4346-967a-0d3ccc120f0c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:43.399580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] The token cache is being invalidate because project 4e57acafb4f14db2a10926c044d458db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:43.399872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:43.400504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-59c46076-3007-4346-967a-0d3ccc120f0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:43.402035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1248/2509] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:43 2022] DELETE /identity/v3/projects/4e57acafb4f14db2a10926c044d458db => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:43.407448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cc88f-8b1c-4808-b4bf-ffb9a37658a2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.407945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cc88f-8b1c-4808-b4bf-ffb9a37658a2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.408119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cc88f-8b1c-4808-b4bf-ffb9a37658a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.426439 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b14cc88f-8b1c-4808-b4bf-ffb9a37658a2 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:43.427642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b14cc88f-8b1c-4808-b4bf-ffb9a37658a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:43.498558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1262/2510] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:41:43.507428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4694f8-4280-480c-8dc2-7c6df4aba0d7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.507781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4694f8-4280-480c-8dc2-7c6df4aba0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.508112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4694f8-4280-480c-8dc2-7c6df4aba0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.526563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1c4694f8-4280-480c-8dc2-7c6df4aba0d7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:43.527823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1c4694f8-4280-480c-8dc2-7c6df4aba0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:43.542004 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1249/2511] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:43.546694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cc88f-8b1c-4808-b4bf-ffb9a37658a2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:43.558012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cc88f-8b1c-4808-b4bf-ffb9a37658a2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:43.563135 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff0106-0d70-4b19-a5eb-1986c1223c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:43.563807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff0106-0d70-4b19-a5eb-1986c1223c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.564092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff0106-0d70-4b19-a5eb-1986c1223c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.564389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff0106-0d70-4b19-a5eb-1986c1223c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.565116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff0106-0d70-4b19-a5eb-1986c1223c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:41:43.565434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff0106-0d70-4b19-a5eb-1986c1223c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:43.567011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff0106-0d70-4b19-a5eb-1986c1223c5f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:43.580011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1263/2512] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:41:43.584478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4694f8-4280-480c-8dc2-7c6df4aba0d7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:43.594118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4694f8-4280-480c-8dc2-7c6df4aba0d7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:43.599350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c9955-2f0b-439e-ab45-60c24f746b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:43.599935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c9955-2f0b-439e-ab45-60c24f746b3b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.600213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c9955-2f0b-439e-ab45-60c24f746b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.600483 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c9955-2f0b-439e-ab45-60c24f746b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.601276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c9955-2f0b-439e-ab45-60c24f746b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:43.603160 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c9955-2f0b-439e-ab45-60c24f746b3b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:43.633479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1250/2513] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:41:43 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:41:43.638304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff0106-0d70-4b19-a5eb-1986c1223c5f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:43.648577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff0106-0d70-4b19-a5eb-1986c1223c5f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:43.711289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570de8a-7b24-4177-90e6-457df83abb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:43.711978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570de8a-7b24-4177-90e6-457df83abb9e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.712722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570de8a-7b24-4177-90e6-457df83abb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.712722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570de8a-7b24-4177-90e6-457df83abb9e admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.713598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570de8a-7b24-4177-90e6-457df83abb9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:43.715984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570de8a-7b24-4177-90e6-457df83abb9e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:43.744415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1264/2514] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:41:43 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:41:43.748555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c9955-2f0b-439e-ab45-60c24f746b3b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:43.758723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c9955-2f0b-439e-ab45-60c24f746b3b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:43.763568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8899aff-f69b-4949-baba-a9349fb627c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:43.764253 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899aff-f69b-4949-baba-a9349fb627c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.764500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899aff-f69b-4949-baba-a9349fb627c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.764840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899aff-f69b-4949-baba-a9349fb627c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.771580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8899aff-f69b-4949-baba-a9349fb627c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:43.773261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8899aff-f69b-4949-baba-a9349fb627c3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:43.784314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1251/2515] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:41:43.788492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570de8a-7b24-4177-90e6-457df83abb9e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:43.799999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570de8a-7b24-4177-90e6-457df83abb9e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:43.805239 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cf65c-7cf7-4947-af21-795e15ed4d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:43.805927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf65c-7cf7-4947-af21-795e15ed4d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.806214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf65c-7cf7-4947-af21-795e15ed4d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.806492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf65c-7cf7-4947-af21-795e15ed4d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.807231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cf65c-7cf7-4947-af21-795e15ed4d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:43.808918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cf65c-7cf7-4947-af21-795e15ed4d7a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:43.820761 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1265/2516] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:43.825393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8899aff-f69b-4949-baba-a9349fb627c3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:43.835509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8899aff-f69b-4949-baba-a9349fb627c3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:43.843776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5dda8-51a9-41df-b572-82543bf77e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:43.845776 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5dda8-51a9-41df-b572-82543bf77e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.846248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5dda8-51a9-41df-b572-82543bf77e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.846712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5dda8-51a9-41df-b572-82543bf77e97 admin admin] PATH_INFO: `/v3/projects/db1c88ca51364255adcf623d98eebb48/users/f0829f6a0f4f43dd9f0628c67d43c0dc/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.869226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5dda8-51a9-41df-b572-82543bf77e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1c88ca51364255adcf623d98eebb48, user_id=f0829f6a0f4f43dd9f0628c67d43c0dc, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:43.871922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5dda8-51a9-41df-b572-82543bf77e97 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:43.884898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:43.891479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1252/2517] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:43 2022] PUT /identity/v3/projects/db1c88ca51364255adcf623d98eebb48/users/f0829f6a0f4f43dd9f0628c67d43c0dc/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:43.902883 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9ff5f-6484-4016-90a8-8dbbf14ec832 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.903429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9ff5f-6484-4016-90a8-8dbbf14ec832 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.903959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9ff5f-6484-4016-90a8-8dbbf14ec832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.918415 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83243cbe-2165-4552-9f6f-161f41739331 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.918730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83243cbe-2165-4552-9f6f-161f41739331 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.919244 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83243cbe-2165-4552-9f6f-161f41739331 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.921126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1253/2518] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:41:43.927700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b8659-7b76-4f4c-8cd2-1bb9df4b82af None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:43.928088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b8659-7b76-4f4c-8cd2-1bb9df4b82af None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:43.928439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b8659-7b76-4f4c-8cd2-1bb9df4b82af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:43.933478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-39e9ff5f-6484-4016-90a8-8dbbf14ec832 None None] MFA Rules not processed for user `f0829f6a0f4f43dd9f0628c67d43c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:43.934773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-39e9ff5f-6484-4016-90a8-8dbbf14ec832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:43.956930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:43.969236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a26b8659-7b76-4f4c-8cd2-1bb9df4b82af None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:43.970893 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a26b8659-7b76-4f4c-8cd2-1bb9df4b82af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.009400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1266/2519] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:44.014380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9ff5f-6484-4016-90a8-8dbbf14ec832 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.054395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1254/2520] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:41:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:44.066059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b8659-7b76-4f4c-8cd2-1bb9df4b82af None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.067528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a26b8659-7b76-4f4c-8cd2-1bb9df4b82af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.085903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9ff5f-6484-4016-90a8-8dbbf14ec832 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.090488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a7f5a-44a1-4886-b186-fbc3efd73ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.091155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a7f5a-44a1-4886-b186-fbc3efd73ace admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.091438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a7f5a-44a1-4886-b186-fbc3efd73ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.091716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a7f5a-44a1-4886-b186-fbc3efd73ace admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.092406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a7f5a-44a1-4886-b186-fbc3efd73ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.094702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a7f5a-44a1-4886-b186-fbc3efd73ace admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.124450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1267/2521] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:41:44 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:44.125457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a7f5a-44a1-4886-b186-fbc3efd73ace admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.147671 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b8659-7b76-4f4c-8cd2-1bb9df4b82af None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.189945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a7f5a-44a1-4886-b186-fbc3efd73ace admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.194676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473ed6c-1519-4bed-aaba-faa055c228a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.195345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473ed6c-1519-4bed-aaba-faa055c228a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.195627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473ed6c-1519-4bed-aaba-faa055c228a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.195907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473ed6c-1519-4bed-aaba-faa055c228a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.196595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473ed6c-1519-4bed-aaba-faa055c228a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.198444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c7235-ec06-47c0-9ac7-098f39d02c71 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b75a27126c9464989cec70e5104824c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.198576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473ed6c-1519-4bed-aaba-faa055c228a2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.199099 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c7235-ec06-47c0-9ac7-098f39d02c71 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.199387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c7235-ec06-47c0-9ac7-098f39d02c71 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.199664 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c7235-ec06-47c0-9ac7-098f39d02c71 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.224884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1268/2522] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:41:44 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:41:44.226141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473ed6c-1519-4bed-aaba-faa055c228a2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.229974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:44.235935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473ed6c-1519-4bed-aaba-faa055c228a2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.240571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169fc30-4c7f-4992-97ea-2d3f8b1c6e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.241260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169fc30-4c7f-4992-97ea-2d3f8b1c6e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.241549 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169fc30-4c7f-4992-97ea-2d3f8b1c6e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.241824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169fc30-4c7f-4992-97ea-2d3f8b1c6e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.242545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169fc30-4c7f-4992-97ea-2d3f8b1c6e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.244130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169fc30-4c7f-4992-97ea-2d3f8b1c6e05 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.254102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1269/2523] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:41:44.258117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169fc30-4c7f-4992-97ea-2d3f8b1c6e05 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.267991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169fc30-4c7f-4992-97ea-2d3f8b1c6e05 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.271031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c7235-ec06-47c0-9ac7-098f39d02c71 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.272470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c7235-ec06-47c0-9ac7-098f39d02c71 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.272838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba79311-9aa7-4a79-8c5c-56b6117222c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.273728 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba79311-9aa7-4a79-8c5c-56b6117222c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.274011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba79311-9aa7-4a79-8c5c-56b6117222c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.274287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba79311-9aa7-4a79-8c5c-56b6117222c9 admin admin] PATH_INFO: `/v3/projects/e65d5a107ea341d382649e5130ffcfc5/users/14bf06627e7648648d162de5eb7cb536/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.285398 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1255/2524] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:41:44.290277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba79311-9aa7-4a79-8c5c-56b6117222c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e65d5a107ea341d382649e5130ffcfc5, user_id=14bf06627e7648648d162de5eb7cb536, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.293319 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba79311-9aa7-4a79-8c5c-56b6117222c9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.310746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1270/2525] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:44 2022] PUT /identity/v3/projects/e65d5a107ea341d382649e5130ffcfc5/users/14bf06627e7648648d162de5eb7cb536/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:41:44.315242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c7235-ec06-47c0-9ac7-098f39d02c71 service glance] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.327035 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c7235-ec06-47c0-9ac7-098f39d02c71 service glance] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.355338 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3aac5-1404-444f-805f-0284e867f834 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.355625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3aac5-1404-444f-805f-0284e867f834 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.355902 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3aac5-1404-444f-805f-0284e867f834 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.357803 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1271/2526] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:41:44.362219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a454e5-0e39-48d9-a244-bb6efd52e703 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.362580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a454e5-0e39-48d9-a244-bb6efd52e703 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.362927 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a454e5-0e39-48d9-a244-bb6efd52e703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.369689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:44.385255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05589d8-27a7-4bc7-a686-639cdefd01cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.385462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-36a454e5-0e39-48d9-a244-bb6efd52e703 None None] MFA Rules not processed for user `a86de22b0e324c70aad792ec6b1a7e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:44.385854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05589d8-27a7-4bc7-a686-639cdefd01cc admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.386133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05589d8-27a7-4bc7-a686-639cdefd01cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.386404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05589d8-27a7-4bc7-a686-639cdefd01cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.386552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-36a454e5-0e39-48d9-a244-bb6efd52e703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.387130 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05589d8-27a7-4bc7-a686-639cdefd01cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.389019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05589d8-27a7-4bc7-a686-639cdefd01cc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.399472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1256/2527] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:44.403692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05589d8-27a7-4bc7-a686-639cdefd01cc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.413500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05589d8-27a7-4bc7-a686-639cdefd01cc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.418460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3bb16-2f21-4651-bf90-8ff7f0bd06e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.419367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bb16-2f21-4651-bf90-8ff7f0bd06e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.419649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bb16-2f21-4651-bf90-8ff7f0bd06e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.419928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bb16-2f21-4651-bf90-8ff7f0bd06e6 admin admin] PATH_INFO: `/v3/projects/e65d5a107ea341d382649e5130ffcfc5/users/14bf06627e7648648d162de5eb7cb536/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.421990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3bb16-2f21-4651-bf90-8ff7f0bd06e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e65d5a107ea341d382649e5130ffcfc5, user_id=14bf06627e7648648d162de5eb7cb536, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.425371 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3bb16-2f21-4651-bf90-8ff7f0bd06e6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.436871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1257/2528] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:44 2022] PUT /identity/v3/projects/e65d5a107ea341d382649e5130ffcfc5/users/14bf06627e7648648d162de5eb7cb536/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:41:44.443116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1272/2529] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:41:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:41:44.447078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a0ece-fbf7-4089-b7b5-f7681d8c724f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.447370 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a0ece-fbf7-4089-b7b5-f7681d8c724f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.447650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a0ece-fbf7-4089-b7b5-f7681d8c724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.456256 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a454e5-0e39-48d9-a244-bb6efd52e703 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.457762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-36a454e5-0e39-48d9-a244-bb6efd52e703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.468536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a454e5-0e39-48d9-a244-bb6efd52e703 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.476889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d33a0ece-fbf7-4089-b7b5-f7681d8c724f None None] MFA Rules not processed for user `14bf06627e7648648d162de5eb7cb536`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:44.478306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d33a0ece-fbf7-4089-b7b5-f7681d8c724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.517428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398296-1602-4d47-a328-7577a4c8cff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.518088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398296-1602-4d47-a328-7577a4c8cff4 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.518371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398296-1602-4d47-a328-7577a4c8cff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.518650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398296-1602-4d47-a328-7577a4c8cff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.527252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398296-1602-4d47-a328-7577a4c8cff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.529221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398296-1602-4d47-a328-7577a4c8cff4 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.538734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1273/2530] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:44.552344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1258/2531] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:44.563221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398296-1602-4d47-a328-7577a4c8cff4 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.564658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ef398296-1602-4d47-a328-7577a4c8cff4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.623933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa941237-a166-4a0a-9b9b-4dfc01a0fb30 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.624209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa941237-a166-4a0a-9b9b-4dfc01a0fb30 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.624490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa941237-a166-4a0a-9b9b-4dfc01a0fb30 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.625904 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398296-1602-4d47-a328-7577a4c8cff4 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.626024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1259/2532] 10.222.0.69 () {54 vars in 921 bytes} [Fri Jun 24 10:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:41:44.629732 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2c39f-40c9-40dd-b6ff-109cf9290006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.630408 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2c39f-40c9-40dd-b6ff-109cf9290006 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.630689 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2c39f-40c9-40dd-b6ff-109cf9290006 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.630967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2c39f-40c9-40dd-b6ff-109cf9290006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.631413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce88f4-5e1f-45e3-9785-27976bd0c276 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.631750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce88f4-5e1f-45e3-9785-27976bd0c276 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.632033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce88f4-5e1f-45e3-9785-27976bd0c276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.641205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2c39f-40c9-40dd-b6ff-109cf9290006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.642692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2c39f-40c9-40dd-b6ff-109cf9290006 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.650154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-12ce88f4-5e1f-45e3-9785-27976bd0c276 None None] MFA Rules not processed for user `bec9fee282a64933aa0f6fbdd91a93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:44.651420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-12ce88f4-5e1f-45e3-9785-27976bd0c276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.652297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1274/2533] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:44.672045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:44.723268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1260/2534] 10.222.0.69 () {58 vars in 1018 bytes} [Fri Jun 24 10:41:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:41:44.735718 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2c39f-40c9-40dd-b6ff-109cf9290006 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.737234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c6b2c39f-40c9-40dd-b6ff-109cf9290006 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.753573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2c39f-40c9-40dd-b6ff-109cf9290006 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.757571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f16ee-4feb-4fb3-9bc4-627634358f3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.758218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f16ee-4feb-4fb3-9bc4-627634358f3f service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.758480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f16ee-4feb-4fb3-9bc4-627634358f3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.758765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f16ee-4feb-4fb3-9bc4-627634358f3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.772910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f16ee-4feb-4fb3-9bc4-627634358f3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.774618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f16ee-4feb-4fb3-9bc4-627634358f3f service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.789640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1275/2535] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:41:44.859342 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce88f4-5e1f-45e3-9785-27976bd0c276 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:44.860806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-12ce88f4-5e1f-45e3-9785-27976bd0c276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.936848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce88f4-5e1f-45e3-9785-27976bd0c276 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:44.942226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e350e-7d8e-48c1-b4bd-ab457e558385 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:44.942865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e350e-7d8e-48c1-b4bd-ab457e558385 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:44.943154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e350e-7d8e-48c1-b4bd-ab457e558385 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:44.943426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e350e-7d8e-48c1-b4bd-ab457e558385 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:44.945392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-866e350e-7d8e-48c1-b4bd-ab457e558385 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:44.957678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e350e-7d8e-48c1-b4bd-ab457e558385 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:44.959249 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e350e-7d8e-48c1-b4bd-ab457e558385 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:44.971759 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1261/2536] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:45.025182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f16ee-4feb-4fb3-9bc4-627634358f3f service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:45.075123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:45.103111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f16ee-4feb-4fb3-9bc4-627634358f3f service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:45.109232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:45.110140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:45.110432 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:45.110696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] PATH_INFO: `/v3/users/f0829f6a0f4f43dd9f0628c67d43c0dc` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:45.112251 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0829f6a0f4f43dd9f0628c67d43c0dc)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:45.114390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:45.172366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0829f6a0f4f43dd9f0628c67d43c0dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.184707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0829f6a0f4f43dd9f0628c67d43c0dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.198493 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0829f6a0f4f43dd9f0628c67d43c0dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.199355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0829f6a0f4f43dd9f0628c67d43c0dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.209707 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0829f6a0f4f43dd9f0628c67d43c0dc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.219213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1276/2537] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:45 2022] DELETE /identity/v3/users/f0829f6a0f4f43dd9f0628c67d43c0dc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:45.223935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550a86d-203b-4113-90ed-47c1ded56819 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:45.224221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550a86d-203b-4113-90ed-47c1ded56819 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:45.224502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550a86d-203b-4113-90ed-47c1ded56819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:45.246526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d550a86d-203b-4113-90ed-47c1ded56819 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:45.248760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d550a86d-203b-4113-90ed-47c1ded56819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:45.320597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1262/2538] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:45.330489 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:45.331907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:45.398735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59320c-dc76-4c6b-8f92-5d9381eeb8ce admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:45.402265 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b01a54-b624-4fc6-a0ce-7440b11a416b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:45.402975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b01a54-b624-4fc6-a0ce-7440b11a416b service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:45.403264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b01a54-b624-4fc6-a0ce-7440b11a416b service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:45.403543 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b01a54-b624-4fc6-a0ce-7440b11a416b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:45.413297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b01a54-b624-4fc6-a0ce-7440b11a416b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:45.414754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b01a54-b624-4fc6-a0ce-7440b11a416b service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:45.424537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1277/2539] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:45.716565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550a86d-203b-4113-90ed-47c1ded56819 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:45.718102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d550a86d-203b-4113-90ed-47c1ded56819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:45.731098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550a86d-203b-4113-90ed-47c1ded56819 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:45.738950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:45.739880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:45.740138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:45.740438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] PATH_INFO: `/v3/projects/db1c88ca51364255adcf623d98eebb48` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:45.741673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1c88ca51364255adcf623d98eebb48)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:45.743636 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:45.788185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:45.795449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1c88ca51364255adcf623d98eebb48', 'request_id': 'req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.802111 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1c88ca51364255adcf623d98eebb48', 'request_id': 'req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.814170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1c88ca51364255adcf623d98eebb48', 'request_id': 'req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.826882 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] The token cache is being invalidate because project db1c88ca51364255adcf623d98eebb48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:45.827439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.828276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.830114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1263/2540] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:45 2022] DELETE /identity/v3/projects/db1c88ca51364255adcf623d98eebb48 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:45.834455 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b01a54-b624-4fc6-a0ce-7440b11a416b service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:45.845020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b01a54-b624-4fc6-a0ce-7440b11a416b service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:45.912016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:45.912838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:45.913124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:45.913406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] PATH_INFO: `/v3/users/14bf06627e7648648d162de5eb7cb536` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:45.914599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] RBAC: Authorizing `identity:delete_user(user_id=14bf06627e7648648d162de5eb7cb536)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:45.916579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:45.922342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:45.968080 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14bf06627e7648648d162de5eb7cb536'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.981680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14bf06627e7648648d162de5eb7cb536'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.990361 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14bf06627e7648648d162de5eb7cb536'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.990985 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14bf06627e7648648d162de5eb7cb536'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:45.998342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-594aa4b8-3e02-4205-961e-973ede3aeadc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14bf06627e7648648d162de5eb7cb536'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:46.004619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1278/2541] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:45 2022] DELETE /identity/v3/users/14bf06627e7648648d162de5eb7cb536 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:46.287041 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:46.288840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:46.323427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d7958-70fb-4fe9-9d5d-18b0a2dae93b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:46.389395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:46.390278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.390561 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.390838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] PATH_INFO: `/v3/projects/e65d5a107ea341d382649e5130ffcfc5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.392026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e65d5a107ea341d382649e5130ffcfc5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:46.393988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:46.458524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e65d5a107ea341d382649e5130ffcfc5', 'request_id': 'req-18eea4ed-1d4b-4b36-850f-544b360f3223'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:46.466056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e65d5a107ea341d382649e5130ffcfc5', 'request_id': 'req-18eea4ed-1d4b-4b36-850f-544b360f3223'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:46.475556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e65d5a107ea341d382649e5130ffcfc5', 'request_id': 'req-18eea4ed-1d4b-4b36-850f-544b360f3223'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:46.486439 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] The token cache is being invalidate because project e65d5a107ea341d382649e5130ffcfc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:46.486730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:46.487448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-18eea4ed-1d4b-4b36-850f-544b360f3223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:46.489096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1264/2542] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:46 2022] DELETE /identity/v3/projects/e65d5a107ea341d382649e5130ffcfc5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:46.495820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5405-24aa-4682-b667-c68588dfe84d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.496113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5405-24aa-4682-b667-c68588dfe84d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.496500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5405-24aa-4682-b667-c68588dfe84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.503262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644785e-fe33-4f35-8f3b-fc434d0ecdb8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.503623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644785e-fe33-4f35-8f3b-fc434d0ecdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.503904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644785e-fe33-4f35-8f3b-fc434d0ecdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.514817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-19ab5405-24aa-4682-b667-c68588dfe84d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:46.516098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-19ab5405-24aa-4682-b667-c68588dfe84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:46.522634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8644785e-fe33-4f35-8f3b-fc434d0ecdb8 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:46.523808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8644785e-fe33-4f35-8f3b-fc434d0ecdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:46.575993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1279/2543] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:46.581141 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1265/2544] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:41:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:41:46.583801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac542b2d-c92a-4a9e-ab06-1a538eb44033 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.584096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac542b2d-c92a-4a9e-ab06-1a538eb44033 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.584370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac542b2d-c92a-4a9e-ab06-1a538eb44033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.593214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644785e-fe33-4f35-8f3b-fc434d0ecdb8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:46.594927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8644785e-fe33-4f35-8f3b-fc434d0ecdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:46.601027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:46.604976 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ac542b2d-c92a-4a9e-ab06-1a538eb44033 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:46.606537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ac542b2d-c92a-4a9e-ab06-1a538eb44033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:46.616816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1280/2545] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:46.621087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac542b2d-c92a-4a9e-ab06-1a538eb44033 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:46.630513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac542b2d-c92a-4a9e-ab06-1a538eb44033 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:46.635917 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55518b37-329a-470d-bb72-286148c63588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:46.636575 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55518b37-329a-470d-bb72-286148c63588 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.636920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55518b37-329a-470d-bb72-286148c63588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.637227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55518b37-329a-470d-bb72-286148c63588 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.638037 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55518b37-329a-470d-bb72-286148c63588 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:41:46.638345 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55518b37-329a-470d-bb72-286148c63588 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:46.639972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55518b37-329a-470d-bb72-286148c63588 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:46.656965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1281/2546] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:41:46.658391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55518b37-329a-470d-bb72-286148c63588 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:46.668047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55518b37-329a-470d-bb72-286148c63588 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:46.673026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade27625-2c35-4603-8efd-5c866a566e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:46.673692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade27625-2c35-4603-8efd-5c866a566e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.674020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade27625-2c35-4603-8efd-5c866a566e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.674238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade27625-2c35-4603-8efd-5c866a566e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.674955 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade27625-2c35-4603-8efd-5c866a566e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:46.675221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644785e-fe33-4f35-8f3b-fc434d0ecdb8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:46.676900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade27625-2c35-4603-8efd-5c866a566e24 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:46.680185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807f49e-e71e-4234-ab9c-ffce65506882 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:46.680880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807f49e-e71e-4234-ab9c-ffce65506882 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.681166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807f49e-e71e-4234-ab9c-ffce65506882 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.681443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807f49e-e71e-4234-ab9c-ffce65506882 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.689931 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807f49e-e71e-4234-ab9c-ffce65506882 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:46.691436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807f49e-e71e-4234-ab9c-ffce65506882 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:46.700680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1266/2547] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:41:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 10:41:46.706263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1282/2548] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:41:46 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:41:46.710293 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807f49e-e71e-4234-ab9c-ffce65506882 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:46.721463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807f49e-e71e-4234-ab9c-ffce65506882 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:46.783456 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e9e54-3b3d-4aa0-9d19-0e7032768d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:46.784235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e9e54-3b3d-4aa0-9d19-0e7032768d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.785208 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e9e54-3b3d-4aa0-9d19-0e7032768d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.785563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e9e54-3b3d-4aa0-9d19-0e7032768d36 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.786346 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e9e54-3b3d-4aa0-9d19-0e7032768d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:46.788828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e9e54-3b3d-4aa0-9d19-0e7032768d36 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:46.821339 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1267/2549] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:41:46 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:41:46.823128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade27625-2c35-4603-8efd-5c866a566e24 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:46.841500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade27625-2c35-4603-8efd-5c866a566e24 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:46.847159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68115985-9af9-4cf8-ac50-2e8daba43f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:46.847971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68115985-9af9-4cf8-ac50-2e8daba43f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.848412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68115985-9af9-4cf8-ac50-2e8daba43f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.848873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68115985-9af9-4cf8-ac50-2e8daba43f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.850713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68115985-9af9-4cf8-ac50-2e8daba43f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:46.851794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68115985-9af9-4cf8-ac50-2e8daba43f67 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:46.868046 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1283/2550] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:41:46.871904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e9e54-3b3d-4aa0-9d19-0e7032768d36 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:46.890070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e9e54-3b3d-4aa0-9d19-0e7032768d36 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:46.905528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d7be6-e274-4ffd-b1bb-2aaaf0dde5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:46.906718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d7be6-e274-4ffd-b1bb-2aaaf0dde5cd admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.907142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d7be6-e274-4ffd-b1bb-2aaaf0dde5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.907594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d7be6-e274-4ffd-b1bb-2aaaf0dde5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:46.908650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d7be6-e274-4ffd-b1bb-2aaaf0dde5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:46.913117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d7be6-e274-4ffd-b1bb-2aaaf0dde5cd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:46.943864 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1268/2551] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:41:46.954704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68115985-9af9-4cf8-ac50-2e8daba43f67 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:46.969853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68115985-9af9-4cf8-ac50-2e8daba43f67 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:46.976449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c9dd3-ec85-4da8-b41c-8ea8164733b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:46.977958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c9dd3-ec85-4da8-b41c-8ea8164733b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:46.978480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c9dd3-ec85-4da8-b41c-8ea8164733b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:46.979676 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c9dd3-ec85-4da8-b41c-8ea8164733b9 admin admin] PATH_INFO: `/v3/projects/875208210e334c11a753c9f0fb9194d0/users/d8e96e06049a400f9a2c4e8a171c1cb2/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:47.001051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c9dd3-ec85-4da8-b41c-8ea8164733b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=875208210e334c11a753c9f0fb9194d0, user_id=d8e96e06049a400f9a2c4e8a171c1cb2, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:47.004268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c9dd3-ec85-4da8-b41c-8ea8164733b9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:47.024581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1284/2552] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:41:46 2022] PUT /identity/v3/projects/875208210e334c11a753c9f0fb9194d0/users/d8e96e06049a400f9a2c4e8a171c1cb2/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:47.037715 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98266b06-a339-487c-89dc-d39c021aa52d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:47.038145 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98266b06-a339-487c-89dc-d39c021aa52d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:47.038518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98266b06-a339-487c-89dc-d39c021aa52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:47.073652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-98266b06-a339-487c-89dc-d39c021aa52d None None] MFA Rules not processed for user `d8e96e06049a400f9a2c4e8a171c1cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:47.074937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-98266b06-a339-487c-89dc-d39c021aa52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:47.155577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1269/2553] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:47.174650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c9dd3-ec85-4da8-b41c-8ea8164733b9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:47.177630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0b5c9dd3-ec85-4da8-b41c-8ea8164733b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:47.189121 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:47.251096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c9dd3-ec85-4da8-b41c-8ea8164733b9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:47.258334 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c850e91-4ca7-4dae-bdee-31cf40bee337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:47.259520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c850e91-4ca7-4dae-bdee-31cf40bee337 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:47.260030 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c850e91-4ca7-4dae-bdee-31cf40bee337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:47.260461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c850e91-4ca7-4dae-bdee-31cf40bee337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:47.272518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c850e91-4ca7-4dae-bdee-31cf40bee337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:47.274053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c850e91-4ca7-4dae-bdee-31cf40bee337 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:47.283290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1285/2554] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:57.152148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:57.279119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=22 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:41:57.581050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98266b06-a339-487c-89dc-d39c021aa52d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:57.647805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98266b06-a339-487c-89dc-d39c021aa52d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:57.652860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:57.653677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:57.653955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:57.654224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] PATH_INFO: `/v3/users/d8e96e06049a400f9a2c4e8a171c1cb2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:57.655449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e96e06049a400f9a2c4e8a171c1cb2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:57.657649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:57.709704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e96e06049a400f9a2c4e8a171c1cb2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:57.722629 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e96e06049a400f9a2c4e8a171c1cb2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:57.731147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e96e06049a400f9a2c4e8a171c1cb2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:57.738424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e96e06049a400f9a2c4e8a171c1cb2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:57.739066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e96e06049a400f9a2c4e8a171c1cb2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:57.745000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1270/2555] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:41:57 2022] DELETE /identity/v3/users/d8e96e06049a400f9a2c4e8a171c1cb2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:41:57.749573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6532b7-e8d2-4b1b-8970-c0133373f041 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:57.749865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6532b7-e8d2-4b1b-8970-c0133373f041 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:57.750144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6532b7-e8d2-4b1b-8970-c0133373f041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:57.768147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fc6532b7-e8d2-4b1b-8970-c0133373f041 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:41:57.769393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fc6532b7-e8d2-4b1b-8970-c0133373f041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:57.840520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1286/2556] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:41:57.851006 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:57.852709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:57.914969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc59fe-4600-4f3d-b677-16e69a621924 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:57.919102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb7726-6d24-4532-a8b0-2630ade16027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:57.919781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7726-6d24-4532-a8b0-2630ade16027 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:57.920050 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7726-6d24-4532-a8b0-2630ade16027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:57.920334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7726-6d24-4532-a8b0-2630ade16027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:57.930295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb7726-6d24-4532-a8b0-2630ade16027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:57.931719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb7726-6d24-4532-a8b0-2630ade16027 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:57.941407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1271/2557] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:41:58.218161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6532b7-e8d2-4b1b-8970-c0133373f041 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:41:58.220154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fc6532b7-e8d2-4b1b-8970-c0133373f041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:41:58.233129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6532b7-e8d2-4b1b-8970-c0133373f041 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:41:58.238583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:41:58.239572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:41:58.239917 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:41:58.240289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] PATH_INFO: `/v3/projects/875208210e334c11a753c9f0fb9194d0` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:41:58.241665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=875208210e334c11a753c9f0fb9194d0)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:41:58.243636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:41:58.296892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '875208210e334c11a753c9f0fb9194d0', 'request_id': 'req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:58.306362 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '875208210e334c11a753c9f0fb9194d0', 'request_id': 'req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:58.312055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '875208210e334c11a753c9f0fb9194d0', 'request_id': 'req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:58.323552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] The token cache is being invalidate because project 875208210e334c11a753c9f0fb9194d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:41:58.323841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:58.324502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed434e0d-2f34-41ad-aef1-2ef3e6f8e5d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:41:58.326104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1287/2558] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:41:58 2022] DELETE /identity/v3/projects/875208210e334c11a753c9f0fb9194d0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:42:00.141048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb7726-6d24-4532-a8b0-2630ade16027 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:00.142657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-21eb7726-6d24-4532-a8b0-2630ade16027 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:00.176525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:00.238307 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb7726-6d24-4532-a8b0-2630ade16027 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:00.358345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:00.359382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:00.364969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:00.366188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:00.367900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:00.386245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:00.387922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:00.400356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1272/2559] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:01.774224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c753f7-f7bf-46b6-9cec-c904e0f83414 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:01.774710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c753f7-f7bf-46b6-9cec-c904e0f83414 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:01.774888 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c753f7-f7bf-46b6-9cec-c904e0f83414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:01.793807 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-80c753f7-f7bf-46b6-9cec-c904e0f83414 None None] MFA Rules not processed for user `5fc073d480884782b4084952345382b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:01.795051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-80c753f7-f7bf-46b6-9cec-c904e0f83414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:01.852101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1288/2560] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:42:02.166445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:02.168000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:02.226718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cdf7a-2886-4167-9043-73d736aae0a1 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:02.230533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:02.231174 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:02.231453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:02.231724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:02.233581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:02.243580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:02.245043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:02.254240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1273/2561] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:02.621805 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0979d-7534-4fb9-bf3b-f628a6c9f6a6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:02.622094 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0979d-7534-4fb9-bf3b-f628a6c9f6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:02.622377 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0979d-7534-4fb9-bf3b-f628a6c9f6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:02.640719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-76f0979d-7534-4fb9-bf3b-f628a6c9f6a6 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:02.641918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-76f0979d-7534-4fb9-bf3b-f628a6c9f6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:02.671048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:02.712929 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1289/2562] 10.222.0.69 () {58 vars in 967 bytes} [Fri Jun 24 10:42:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:02.725467 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:02.727163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:02.795599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:02.835059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b847d0-8a7e-491d-bbae-fe875b5a3661 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:02.840296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf496-dc41-444d-9d8c-e80a3497da50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:02.840980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf496-dc41-444d-9d8c-e80a3497da50 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:02.841268 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf496-dc41-444d-9d8c-e80a3497da50 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:02.841548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf496-dc41-444d-9d8c-e80a3497da50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:02.850739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9caf496-dc41-444d-9d8c-e80a3497da50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:02.852335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9caf496-dc41-444d-9d8c-e80a3497da50 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:02.862560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1274/2563] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:42:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:42:06.818161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0979d-7534-4fb9-bf3b-f628a6c9f6a6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:06.820388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-76f0979d-7534-4fb9-bf3b-f628a6c9f6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:06.845532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0979d-7534-4fb9-bf3b-f628a6c9f6a6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:06.863817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:06.865212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:06.865212 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:06.865454 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:06.869669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:06.901790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:06.903448 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:06.924069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1290/2564] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:11.163059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf496-dc41-444d-9d8c-e80a3497da50 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:11.172728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf496-dc41-444d-9d8c-e80a3497da50 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:11.176563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def3228-ee4f-4ccc-82ae-afdedc2dc2c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:11.177208 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def3228-ee4f-4ccc-82ae-afdedc2dc2c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:11.177492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def3228-ee4f-4ccc-82ae-afdedc2dc2c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:11.177740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def3228-ee4f-4ccc-82ae-afdedc2dc2c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:11.179724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0def3228-ee4f-4ccc-82ae-afdedc2dc2c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:11.244831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def3228-ee4f-4ccc-82ae-afdedc2dc2c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:11.246196 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def3228-ee4f-4ccc-82ae-afdedc2dc2c0 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:11.256369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1275/2565] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:14.644102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:14.645734 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:14.659816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7fef9-54a7-4268-9d5d-fecffbe8e308 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:14.664778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:14.665650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:14.665931 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:14.666206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] PATH_INFO: `/v3/users/733b7d5058da45089a2133e2fe4695e7` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:14.667375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=733b7d5058da45089a2133e2fe4695e7)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:14.669386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:14.721097 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733b7d5058da45089a2133e2fe4695e7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:14.733799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733b7d5058da45089a2133e2fe4695e7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:14.744519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733b7d5058da45089a2133e2fe4695e7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:14.747716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733b7d5058da45089a2133e2fe4695e7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:14.753101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-53263ba3-53b0-4c41-aacd-993b60e543f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733b7d5058da45089a2133e2fe4695e7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:14.759466 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1291/2566] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:14 2022] DELETE /identity/v3/users/733b7d5058da45089a2133e2fe4695e7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:15.021621 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def3228-ee4f-4ccc-82ae-afdedc2dc2c0 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:15.034365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def3228-ee4f-4ccc-82ae-afdedc2dc2c0 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:15.096133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:15.097043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.097326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.097599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] PATH_INFO: `/v3/projects/e41afc594f464d74bd4881e07dd33e45` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.098876 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41afc594f464d74bd4881e07dd33e45)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:15.100905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:15.152350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41afc594f464d74bd4881e07dd33e45', 'request_id': 'req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:15.158655 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41afc594f464d74bd4881e07dd33e45', 'request_id': 'req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:15.168937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41afc594f464d74bd4881e07dd33e45', 'request_id': 'req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:15.180565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] The token cache is being invalidate because project e41afc594f464d74bd4881e07dd33e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:15.180891 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:15.181542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f52f9ce-21d6-42bf-b5af-5ef5b6bf444b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:15.183108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1276/2567] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:15 2022] DELETE /identity/v3/projects/e41afc594f464d74bd4881e07dd33e45 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:15.188300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a1e66-e29f-4287-bd73-2bf21ea2eb74 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.188590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a1e66-e29f-4287-bd73-2bf21ea2eb74 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.188908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a1e66-e29f-4287-bd73-2bf21ea2eb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.207808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b16a1e66-e29f-4287-bd73-2bf21ea2eb74 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:15.209071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b16a1e66-e29f-4287-bd73-2bf21ea2eb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:15.278170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1292/2568] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:15.285922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61b48-1aac-4162-8d23-1cdece328465 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.286214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61b48-1aac-4162-8d23-1cdece328465 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.286499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61b48-1aac-4162-8d23-1cdece328465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.293668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:15.308115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-88d61b48-1aac-4162-8d23-1cdece328465 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:15.310147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-88d61b48-1aac-4162-8d23-1cdece328465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:15.319686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1277/2569] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:15.323847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a1e66-e29f-4287-bd73-2bf21ea2eb74 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:15.333887 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a1e66-e29f-4287-bd73-2bf21ea2eb74 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:15.339289 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c66bb-4f55-4b33-98c7-085b6ba87ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:15.340064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c66bb-4f55-4b33-98c7-085b6ba87ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.340347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c66bb-4f55-4b33-98c7-085b6ba87ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.340625 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c66bb-4f55-4b33-98c7-085b6ba87ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.341394 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c66bb-4f55-4b33-98c7-085b6ba87ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:42:15.341703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c66bb-4f55-4b33-98c7-085b6ba87ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:15.343353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c66bb-4f55-4b33-98c7-085b6ba87ce0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:15.356314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1293/2570] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:42:15.361164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61b48-1aac-4162-8d23-1cdece328465 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:15.371463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61b48-1aac-4162-8d23-1cdece328465 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:15.377933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a4c8-6557-458c-ad94-a47450337a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:15.378587 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a4c8-6557-458c-ad94-a47450337a10 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.378868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a4c8-6557-458c-ad94-a47450337a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.379139 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a4c8-6557-458c-ad94-a47450337a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.379828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3a4c8-6557-458c-ad94-a47450337a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:15.381890 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3a4c8-6557-458c-ad94-a47450337a10 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:15.424140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1278/2571] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:15 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:15.428607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c66bb-4f55-4b33-98c7-085b6ba87ce0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:15.448071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c66bb-4f55-4b33-98c7-085b6ba87ce0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:15.474540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:15.521113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41959471-ddb9-4b0f-b548-f8a47c092dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:15.521785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41959471-ddb9-4b0f-b548-f8a47c092dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.522150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41959471-ddb9-4b0f-b548-f8a47c092dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.522451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41959471-ddb9-4b0f-b548-f8a47c092dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.523148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41959471-ddb9-4b0f-b548-f8a47c092dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:15.525102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41959471-ddb9-4b0f-b548-f8a47c092dd3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:15.547007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1294/2572] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:15 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:15.551256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a4c8-6557-458c-ad94-a47450337a10 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:15.561869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a4c8-6557-458c-ad94-a47450337a10 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:15.566519 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9a76d-dec3-47c5-9769-0d4a6ed50826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:15.567217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9a76d-dec3-47c5-9769-0d4a6ed50826 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.567498 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9a76d-dec3-47c5-9769-0d4a6ed50826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.567773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9a76d-dec3-47c5-9769-0d4a6ed50826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.568507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9a76d-dec3-47c5-9769-0d4a6ed50826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:15.570284 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9a76d-dec3-47c5-9769-0d4a6ed50826 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:15.580253 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1279/2573] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:15.584114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41959471-ddb9-4b0f-b548-f8a47c092dd3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:15.594178 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41959471-ddb9-4b0f-b548-f8a47c092dd3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:15.599449 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c82aa-ef74-4358-8a25-535585a8104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:15.600103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c82aa-ef74-4358-8a25-535585a8104f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.600555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c82aa-ef74-4358-8a25-535585a8104f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.600836 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c82aa-ef74-4358-8a25-535585a8104f admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.601582 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c82aa-ef74-4358-8a25-535585a8104f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:15.603175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c82aa-ef74-4358-8a25-535585a8104f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:15.613305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1295/2574] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:15.617513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9a76d-dec3-47c5-9769-0d4a6ed50826 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:15.628053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9a76d-dec3-47c5-9769-0d4a6ed50826 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:15.633580 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2527ca-cf2e-44c7-929e-b153335cabdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:15.634495 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2527ca-cf2e-44c7-929e-b153335cabdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.634778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2527ca-cf2e-44c7-929e-b153335cabdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.635043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2527ca-cf2e-44c7-929e-b153335cabdb admin admin] PATH_INFO: `/v3/projects/01ab7a18d7a44bb0bcdaea13c7eb16f6/users/cf0e636d3c6947d7b64b08317578d5b8/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.649983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2527ca-cf2e-44c7-929e-b153335cabdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ab7a18d7a44bb0bcdaea13c7eb16f6, user_id=cf0e636d3c6947d7b64b08317578d5b8, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:15.652686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2527ca-cf2e-44c7-929e-b153335cabdb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:15.667648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1280/2575] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:15 2022] PUT /identity/v3/projects/01ab7a18d7a44bb0bcdaea13c7eb16f6/users/cf0e636d3c6947d7b64b08317578d5b8/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:15.678964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1af04d-66b5-43e7-8c4a-6e35f6ce01a5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.679517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1af04d-66b5-43e7-8c4a-6e35f6ce01a5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.679517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1af04d-66b5-43e7-8c4a-6e35f6ce01a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.707570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3f1af04d-66b5-43e7-8c4a-6e35f6ce01a5 None None] MFA Rules not processed for user `cf0e636d3c6947d7b64b08317578d5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:15.708831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f1af04d-66b5-43e7-8c4a-6e35f6ce01a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:15.780563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1296/2576] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:15.785235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2527ca-cf2e-44c7-929e-b153335cabdb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:15.851897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2527ca-cf2e-44c7-929e-b153335cabdb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:15.857720 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54affd6e-0db0-4858-8bb5-8be9ea493f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:15.858383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54affd6e-0db0-4858-8bb5-8be9ea493f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.858663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54affd6e-0db0-4858-8bb5-8be9ea493f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.858937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54affd6e-0db0-4858-8bb5-8be9ea493f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.859622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54affd6e-0db0-4858-8bb5-8be9ea493f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:15.861574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54affd6e-0db0-4858-8bb5-8be9ea493f88 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:15.889984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1281/2577] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:15 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:15.894326 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1af04d-66b5-43e7-8c4a-6e35f6ce01a5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:15.906198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1af04d-66b5-43e7-8c4a-6e35f6ce01a5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:15.963934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3013e-c7d7-4424-aff5-810ddd8c2a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:15.964512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3013e-c7d7-4424-aff5-810ddd8c2a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:15.964820 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3013e-c7d7-4424-aff5-810ddd8c2a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:15.965095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3013e-c7d7-4424-aff5-810ddd8c2a86 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:15.965790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3013e-c7d7-4424-aff5-810ddd8c2a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:15.967697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3013e-c7d7-4424-aff5-810ddd8c2a86 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:15.990115 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1297/2578] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:15 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:15.994262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54affd6e-0db0-4858-8bb5-8be9ea493f88 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:16.004812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54affd6e-0db0-4858-8bb5-8be9ea493f88 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:16.009650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e5df6-9285-4592-832e-91b0c91d0b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:16.010314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e5df6-9285-4592-832e-91b0c91d0b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:16.010592 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e5df6-9285-4592-832e-91b0c91d0b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:16.010865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e5df6-9285-4592-832e-91b0c91d0b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:16.011588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e5df6-9285-4592-832e-91b0c91d0b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:16.013235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e5df6-9285-4592-832e-91b0c91d0b22 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:16.025408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1282/2579] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:16.029765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3013e-c7d7-4424-aff5-810ddd8c2a86 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:16.040529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3013e-c7d7-4424-aff5-810ddd8c2a86 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:16.046238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b085b0-cd5d-4ffa-bfe2-27cad8174357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:16.047142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b085b0-cd5d-4ffa-bfe2-27cad8174357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:16.047421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b085b0-cd5d-4ffa-bfe2-27cad8174357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:16.047719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b085b0-cd5d-4ffa-bfe2-27cad8174357 admin admin] PATH_INFO: `/v3/projects/07a5163e06e246788be796392d7c731e/users/b1095f9832bf44f9adad562ffb9366a1/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:16.065968 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b085b0-cd5d-4ffa-bfe2-27cad8174357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a5163e06e246788be796392d7c731e, user_id=b1095f9832bf44f9adad562ffb9366a1, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:16.068701 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b085b0-cd5d-4ffa-bfe2-27cad8174357 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:16.071173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:16.084581 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1298/2580] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:16 2022] PUT /identity/v3/projects/07a5163e06e246788be796392d7c731e/users/b1095f9832bf44f9adad562ffb9366a1/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:16.089064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e5df6-9285-4592-832e-91b0c91d0b22 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:16.099721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e5df6-9285-4592-832e-91b0c91d0b22 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:16.158135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f16b48-643e-4b5b-aa85-16d93b3cb08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:16.158855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f16b48-643e-4b5b-aa85-16d93b3cb08a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:16.159181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f16b48-643e-4b5b-aa85-16d93b3cb08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:16.159627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f16b48-643e-4b5b-aa85-16d93b3cb08a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:16.160937 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f16b48-643e-4b5b-aa85-16d93b3cb08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:16.164017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f16b48-643e-4b5b-aa85-16d93b3cb08a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:16.170227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:16.175742 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1283/2581] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:42:16.180102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b085b0-cd5d-4ffa-bfe2-27cad8174357 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:16.190423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b085b0-cd5d-4ffa-bfe2-27cad8174357 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:16.195100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3a827-ca42-41d9-bd2f-3ceca6cd2c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:16.195983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3a827-ca42-41d9-bd2f-3ceca6cd2c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:16.196261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3a827-ca42-41d9-bd2f-3ceca6cd2c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:16.196537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3a827-ca42-41d9-bd2f-3ceca6cd2c4a admin admin] PATH_INFO: `/v3/projects/07a5163e06e246788be796392d7c731e/users/b1095f9832bf44f9adad562ffb9366a1/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:16.198591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3a827-ca42-41d9-bd2f-3ceca6cd2c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a5163e06e246788be796392d7c731e, user_id=b1095f9832bf44f9adad562ffb9366a1, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:16.201157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3a827-ca42-41d9-bd2f-3ceca6cd2c4a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:16.212145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1299/2582] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:16 2022] PUT /identity/v3/projects/07a5163e06e246788be796392d7c731e/users/b1095f9832bf44f9adad562ffb9366a1/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:42:16.222983 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f70f6f-ff5d-4137-a940-138aa5ead1e9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:16.223211 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f70f6f-ff5d-4137-a940-138aa5ead1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:16.223485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f70f6f-ff5d-4137-a940-138aa5ead1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:16.251261 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-57f70f6f-ff5d-4137-a940-138aa5ead1e9 None None] MFA Rules not processed for user `b1095f9832bf44f9adad562ffb9366a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:16.252499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-57f70f6f-ff5d-4137-a940-138aa5ead1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:16.328714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1284/2583] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:16.340450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3a827-ca42-41d9-bd2f-3ceca6cd2c4a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:16.342206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f5b3a827-ca42-41d9-bd2f-3ceca6cd2c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:16.400139 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3a827-ca42-41d9-bd2f-3ceca6cd2c4a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:16.404343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef91aa9-9916-4125-a9ff-87b3e73c2e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:16.405028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef91aa9-9916-4125-a9ff-87b3e73c2e87 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:16.405310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef91aa9-9916-4125-a9ff-87b3e73c2e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:16.405563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef91aa9-9916-4125-a9ff-87b3e73c2e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:16.414944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef91aa9-9916-4125-a9ff-87b3e73c2e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:16.416379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef91aa9-9916-4125-a9ff-87b3e73c2e87 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:16.425794 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1300/2584] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:16.495351 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f70f6f-ff5d-4137-a940-138aa5ead1e9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:16.562421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f70f6f-ff5d-4137-a940-138aa5ead1e9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:16.567748 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:16.568624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:16.569146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:16.569420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] PATH_INFO: `/v3/users/cf0e636d3c6947d7b64b08317578d5b8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:16.570842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0e636d3c6947d7b64b08317578d5b8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:16.572804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:16.624350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0e636d3c6947d7b64b08317578d5b8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:16.636647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0e636d3c6947d7b64b08317578d5b8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:16.648615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0e636d3c6947d7b64b08317578d5b8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:16.655834 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0e636d3c6947d7b64b08317578d5b8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:16.656462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0e636d3c6947d7b64b08317578d5b8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:16.662316 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1285/2585] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:16 2022] DELETE /identity/v3/users/cf0e636d3c6947d7b64b08317578d5b8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:16.667118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7c536-d6c5-4963-bae9-a9653aea9134 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:16.667529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7c536-d6c5-4963-bae9-a9653aea9134 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:16.667882 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7c536-d6c5-4963-bae9-a9653aea9134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:16.686602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-49a7c536-d6c5-4963-bae9-a9653aea9134 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:16.687821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-49a7c536-d6c5-4963-bae9-a9653aea9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:16.753885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1301/2586] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:16.762097 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:16.763574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:16.770403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:16.825667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33dce6-7485-48b3-a0e5-fa287462fd5d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:16.829875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09cd96-2d90-4688-87ef-71c4b1b0546c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:16.830533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09cd96-2d90-4688-87ef-71c4b1b0546c service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:16.830815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09cd96-2d90-4688-87ef-71c4b1b0546c service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:16.831087 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09cd96-2d90-4688-87ef-71c4b1b0546c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:16.840842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09cd96-2d90-4688-87ef-71c4b1b0546c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:16.842275 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09cd96-2d90-4688-87ef-71c4b1b0546c service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:16.852886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1286/2587] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:42:17.130998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7c536-d6c5-4963-bae9-a9653aea9134 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:17.132722 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-49a7c536-d6c5-4963-bae9-a9653aea9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:17.146211 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7c536-d6c5-4963-bae9-a9653aea9134 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:17.153714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:17.154712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:17.155061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:17.155578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] PATH_INFO: `/v3/projects/01ab7a18d7a44bb0bcdaea13c7eb16f6` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:17.157397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ab7a18d7a44bb0bcdaea13c7eb16f6)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:17.159441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:17.226126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ab7a18d7a44bb0bcdaea13c7eb16f6', 'request_id': 'req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.238258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ab7a18d7a44bb0bcdaea13c7eb16f6', 'request_id': 'req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.246411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ab7a18d7a44bb0bcdaea13c7eb16f6', 'request_id': 'req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.266573 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] The token cache is being invalidate because project 01ab7a18d7a44bb0bcdaea13c7eb16f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:17.266895 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.267516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.269371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1302/2588] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:17 2022] DELETE /identity/v3/projects/01ab7a18d7a44bb0bcdaea13c7eb16f6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:17.276955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09cd96-2d90-4688-87ef-71c4b1b0546c service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:17.290258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09cd96-2d90-4688-87ef-71c4b1b0546c service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:17.362918 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:17.363812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:17.364092 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:17.364365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] PATH_INFO: `/v3/users/b1095f9832bf44f9adad562ffb9366a1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:17.365599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1095f9832bf44f9adad562ffb9366a1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:17.367889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:17.420093 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1095f9832bf44f9adad562ffb9366a1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.433083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1095f9832bf44f9adad562ffb9366a1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.443850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1095f9832bf44f9adad562ffb9366a1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.452000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1095f9832bf44f9adad562ffb9366a1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.452837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c7cb5e36-4952-4941-99d0-4715c04b1bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1095f9832bf44f9adad562ffb9366a1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.461252 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1287/2589] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:17 2022] DELETE /identity/v3/users/b1095f9832bf44f9adad562ffb9366a1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:17.742730 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:17.744287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:17.757793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dac4f5-1f3a-4a76-b5f6-140f7944dbe8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:17.762072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:17.820504 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:17.821337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:17.821688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:17.821870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] PATH_INFO: `/v3/projects/07a5163e06e246788be796392d7c731e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:17.823210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] RBAC: Authorizing `identity:delete_project(project_id=07a5163e06e246788be796392d7c731e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:17.825217 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:17.874970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07a5163e06e246788be796392d7c731e', 'request_id': 'req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.884340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07a5163e06e246788be796392d7c731e', 'request_id': 'req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.889932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07a5163e06e246788be796392d7c731e', 'request_id': 'req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.901000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] The token cache is being invalidate because project 07a5163e06e246788be796392d7c731e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:17.901269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.901936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea4fa9ec-1f9b-4cea-bdd8-ed8c10db424f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:17.903515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1303/2590] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:17 2022] DELETE /identity/v3/projects/07a5163e06e246788be796392d7c731e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:17.908709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595d95-d00e-47a9-bc9e-2719e5a1e126 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:17.909033 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595d95-d00e-47a9-bc9e-2719e5a1e126 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:17.909314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595d95-d00e-47a9-bc9e-2719e5a1e126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:17.929313 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-13595d95-d00e-47a9-bc9e-2719e5a1e126 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:17.930567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-13595d95-d00e-47a9-bc9e-2719e5a1e126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:17.951658 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:17.995513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1288/2591] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:18.003414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f495d6e-db7f-435d-94f9-49285020baae None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.003698 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f495d6e-db7f-435d-94f9-49285020baae None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.003973 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f495d6e-db7f-435d-94f9-49285020baae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.024171 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1f495d6e-db7f-435d-94f9-49285020baae None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:18.025406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1f495d6e-db7f-435d-94f9-49285020baae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:18.034606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1304/2592] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:18.039063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13595d95-d00e-47a9-bc9e-2719e5a1e126 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.049908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13595d95-d00e-47a9-bc9e-2719e5a1e126 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.054785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f383a19-2c66-447d-87b2-1bd791e84190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.055472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f383a19-2c66-447d-87b2-1bd791e84190 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.055759 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f383a19-2c66-447d-87b2-1bd791e84190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.056034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f383a19-2c66-447d-87b2-1bd791e84190 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.056764 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f383a19-2c66-447d-87b2-1bd791e84190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:42:18.057107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f383a19-2c66-447d-87b2-1bd791e84190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.058717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f383a19-2c66-447d-87b2-1bd791e84190 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.071884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1289/2593] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:42:18.076258 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f495d6e-db7f-435d-94f9-49285020baae None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.086071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f495d6e-db7f-435d-94f9-49285020baae None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.092156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14940562-b7ed-46af-9e9d-615f944081e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.092835 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14940562-b7ed-46af-9e9d-615f944081e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.093114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14940562-b7ed-46af-9e9d-615f944081e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.093390 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14940562-b7ed-46af-9e9d-615f944081e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.094101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14940562-b7ed-46af-9e9d-615f944081e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.096307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14940562-b7ed-46af-9e9d-615f944081e2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.128114 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1305/2594] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:18 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:18.132213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f383a19-2c66-447d-87b2-1bd791e84190 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.143186 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f383a19-2c66-447d-87b2-1bd791e84190 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.212553 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75799c9e-226d-47e6-a61a-3ab50f9b5958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.213264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75799c9e-226d-47e6-a61a-3ab50f9b5958 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.213550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75799c9e-226d-47e6-a61a-3ab50f9b5958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.213829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75799c9e-226d-47e6-a61a-3ab50f9b5958 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.214535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75799c9e-226d-47e6-a61a-3ab50f9b5958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.216459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75799c9e-226d-47e6-a61a-3ab50f9b5958 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.238897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1290/2595] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:18 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:18.243160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14940562-b7ed-46af-9e9d-615f944081e2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.253487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14940562-b7ed-46af-9e9d-615f944081e2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.258791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47873910-bca4-49d5-97b2-a3602adcb067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.259440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47873910-bca4-49d5-97b2-a3602adcb067 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.259754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47873910-bca4-49d5-97b2-a3602adcb067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.260006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47873910-bca4-49d5-97b2-a3602adcb067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.260715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47873910-bca4-49d5-97b2-a3602adcb067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.262344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47873910-bca4-49d5-97b2-a3602adcb067 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.272865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1306/2596] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:18.277001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75799c9e-226d-47e6-a61a-3ab50f9b5958 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.287486 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75799c9e-226d-47e6-a61a-3ab50f9b5958 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.293157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b9fec-2512-4131-a0ff-dd7897227c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.293815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b9fec-2512-4131-a0ff-dd7897227c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.294091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b9fec-2512-4131-a0ff-dd7897227c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.294372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b9fec-2512-4131-a0ff-dd7897227c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.295089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b9fec-2512-4131-a0ff-dd7897227c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.296659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b9fec-2512-4131-a0ff-dd7897227c73 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.307473 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1291/2597] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:18.311908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47873910-bca4-49d5-97b2-a3602adcb067 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.322583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47873910-bca4-49d5-97b2-a3602adcb067 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.328621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0cfcc-231f-42d1-9e8a-9fc32d1e39f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.329564 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0cfcc-231f-42d1-9e8a-9fc32d1e39f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.329845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0cfcc-231f-42d1-9e8a-9fc32d1e39f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.330118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0cfcc-231f-42d1-9e8a-9fc32d1e39f8 admin admin] PATH_INFO: `/v3/projects/3d0cc629be89418d96cfce30ad5b553f/users/580bc306f5a847aea5dd768478ed67ae/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.345499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0cfcc-231f-42d1-9e8a-9fc32d1e39f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0cc629be89418d96cfce30ad5b553f, user_id=580bc306f5a847aea5dd768478ed67ae, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.348162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0cfcc-231f-42d1-9e8a-9fc32d1e39f8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.363181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1307/2598] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:18 2022] PUT /identity/v3/projects/3d0cc629be89418d96cfce30ad5b553f/users/580bc306f5a847aea5dd768478ed67ae/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:42:18.375063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12515a0d-81ba-4365-b4e2-fde444f64ebf None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.375466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12515a0d-81ba-4365-b4e2-fde444f64ebf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.375645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12515a0d-81ba-4365-b4e2-fde444f64ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.405623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-12515a0d-81ba-4365-b4e2-fde444f64ebf None None] MFA Rules not processed for user `580bc306f5a847aea5dd768478ed67ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:18.406943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-12515a0d-81ba-4365-b4e2-fde444f64ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:18.492634 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1292/2599] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:18.497357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0cfcc-231f-42d1-9e8a-9fc32d1e39f8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.531802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:18.560832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0cfcc-231f-42d1-9e8a-9fc32d1e39f8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.565646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770085ba-6312-482e-91e7-ed4366817c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.566303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770085ba-6312-482e-91e7-ed4366817c56 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.566586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770085ba-6312-482e-91e7-ed4366817c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.566861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770085ba-6312-482e-91e7-ed4366817c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.567553 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770085ba-6312-482e-91e7-ed4366817c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.569494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770085ba-6312-482e-91e7-ed4366817c56 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.600172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1308/2600] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:18 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:18.606700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12515a0d-81ba-4365-b4e2-fde444f64ebf None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.621842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12515a0d-81ba-4365-b4e2-fde444f64ebf None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.640489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:18.679566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a6a82-33ee-49f2-9e4a-0a931d8b4bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.680235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a6a82-33ee-49f2-9e4a-0a931d8b4bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.680517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a6a82-33ee-49f2-9e4a-0a931d8b4bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.680822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a6a82-33ee-49f2-9e4a-0a931d8b4bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.681504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a6a82-33ee-49f2-9e4a-0a931d8b4bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.684179 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a6a82-33ee-49f2-9e4a-0a931d8b4bd3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.706771 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1293/2601] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:18 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:18.710873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770085ba-6312-482e-91e7-ed4366817c56 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.722280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770085ba-6312-482e-91e7-ed4366817c56 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.727871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951ef65-a2a7-4745-af3c-398394704fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.728527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951ef65-a2a7-4745-af3c-398394704fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.728842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951ef65-a2a7-4745-af3c-398394704fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.729117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951ef65-a2a7-4745-af3c-398394704fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.729846 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951ef65-a2a7-4745-af3c-398394704fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.731453 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951ef65-a2a7-4745-af3c-398394704fb1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.742926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1309/2602] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:18.747311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a6a82-33ee-49f2-9e4a-0a931d8b4bd3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.757844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a6a82-33ee-49f2-9e4a-0a931d8b4bd3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.763089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cedb3-a76a-42cd-b920-29bfb7a1eacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.763938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cedb3-a76a-42cd-b920-29bfb7a1eacd admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.764220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cedb3-a76a-42cd-b920-29bfb7a1eacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.764494 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cedb3-a76a-42cd-b920-29bfb7a1eacd admin admin] PATH_INFO: `/v3/projects/4de1b86b43df47cd82f1a97df20f7d69/users/02c66b0d13e64e849b725cca11d43e68/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.786118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cedb3-a76a-42cd-b920-29bfb7a1eacd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de1b86b43df47cd82f1a97df20f7d69, user_id=02c66b0d13e64e849b725cca11d43e68, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.791184 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cedb3-a76a-42cd-b920-29bfb7a1eacd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.811833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1294/2603] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:18 2022] PUT /identity/v3/projects/4de1b86b43df47cd82f1a97df20f7d69/users/02c66b0d13e64e849b725cca11d43e68/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:18.816578 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951ef65-a2a7-4745-af3c-398394704fb1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.828393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951ef65-a2a7-4745-af3c-398394704fb1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.890216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3665e3-91c8-4964-8191-9779fcec3cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.890819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3665e3-91c8-4964-8191-9779fcec3cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.891099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3665e3-91c8-4964-8191-9779fcec3cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.891375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3665e3-91c8-4964-8191-9779fcec3cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.892105 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3665e3-91c8-4964-8191-9779fcec3cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.893756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3665e3-91c8-4964-8191-9779fcec3cd8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.904359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1310/2604] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:18.908609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cedb3-a76a-42cd-b920-29bfb7a1eacd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:18.919704 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cedb3-a76a-42cd-b920-29bfb7a1eacd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:18.924878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f1cf9-17c9-4448-a886-1c6941daff8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:18.925765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f1cf9-17c9-4448-a886-1c6941daff8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.926054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f1cf9-17c9-4448-a886-1c6941daff8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.926330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f1cf9-17c9-4448-a886-1c6941daff8d admin admin] PATH_INFO: `/v3/projects/4de1b86b43df47cd82f1a97df20f7d69/users/02c66b0d13e64e849b725cca11d43e68/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.928321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f1cf9-17c9-4448-a886-1c6941daff8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de1b86b43df47cd82f1a97df20f7d69, user_id=02c66b0d13e64e849b725cca11d43e68, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:18.931360 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f1cf9-17c9-4448-a886-1c6941daff8d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:18.942754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1295/2605] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:18 2022] PUT /identity/v3/projects/4de1b86b43df47cd82f1a97df20f7d69/users/02c66b0d13e64e849b725cca11d43e68/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:18.954140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347cd8-bdd8-49b6-a849-0282dbc6a32d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:18.954434 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347cd8-bdd8-49b6-a849-0282dbc6a32d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:18.954699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347cd8-bdd8-49b6-a849-0282dbc6a32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:18.983027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-11347cd8-bdd8-49b6-a849-0282dbc6a32d None None] MFA Rules not processed for user `02c66b0d13e64e849b725cca11d43e68`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:18.984221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-11347cd8-bdd8-49b6-a849-0282dbc6a32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:19.059363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1311/2606] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:19.071114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f1cf9-17c9-4448-a886-1c6941daff8d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:19.073304 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4b7f1cf9-17c9-4448-a886-1c6941daff8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:19.131650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f1cf9-17c9-4448-a886-1c6941daff8d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:19.135703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e9eeb-185a-4706-bb63-e3fbe663abdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:19.136349 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e9eeb-185a-4706-bb63-e3fbe663abdf service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:19.136626 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e9eeb-185a-4706-bb63-e3fbe663abdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:19.136928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e9eeb-185a-4706-bb63-e3fbe663abdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:19.147055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e9eeb-185a-4706-bb63-e3fbe663abdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:19.148662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e9eeb-185a-4706-bb63-e3fbe663abdf service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:19.158008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1296/2607] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:19.209025 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11347cd8-bdd8-49b6-a849-0282dbc6a32d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:19.211316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-11347cd8-bdd8-49b6-a849-0282dbc6a32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:19.224232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11347cd8-bdd8-49b6-a849-0282dbc6a32d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:19.228159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2689a0-ca66-4bcc-941a-284c073fa6e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:19.228793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2689a0-ca66-4bcc-941a-284c073fa6e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:19.229101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2689a0-ca66-4bcc-941a-284c073fa6e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:19.229378 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2689a0-ca66-4bcc-941a-284c073fa6e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:19.257638 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:19.290150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2689a0-ca66-4bcc-941a-284c073fa6e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:19.291588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2689a0-ca66-4bcc-941a-284c073fa6e1 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:19.300980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1312/2608] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:22.626471 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e9eeb-185a-4706-bb63-e3fbe663abdf service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:22.690103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e9eeb-185a-4706-bb63-e3fbe663abdf service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:22.695488 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:22.696358 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:22.696643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:22.696955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] PATH_INFO: `/v3/users/580bc306f5a847aea5dd768478ed67ae` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:22.698185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=580bc306f5a847aea5dd768478ed67ae)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:22.700104 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:22.721726 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:22.753172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '580bc306f5a847aea5dd768478ed67ae'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:22.764880 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '580bc306f5a847aea5dd768478ed67ae'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:22.774709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '580bc306f5a847aea5dd768478ed67ae'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:22.782357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '580bc306f5a847aea5dd768478ed67ae'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:22.782984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '580bc306f5a847aea5dd768478ed67ae'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:22.788842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1297/2609] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:22 2022] DELETE /identity/v3/users/580bc306f5a847aea5dd768478ed67ae => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:22.793528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d930bd0-6dd5-4879-aad1-82d6e8581e1c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:22.794033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d930bd0-6dd5-4879-aad1-82d6e8581e1c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:22.794148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d930bd0-6dd5-4879-aad1-82d6e8581e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:22.812513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0d930bd0-6dd5-4879-aad1-82d6e8581e1c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:22.813763 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0d930bd0-6dd5-4879-aad1-82d6e8581e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:22.888206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1313/2610] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:42:22.898279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:22.899919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:22.959946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc796c0-537d-444a-9af3-069fb2c5ec38 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:22.964501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a0af7-d071-4f5f-85b3-f0f1c0670c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:22.965180 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a0af7-d071-4f5f-85b3-f0f1c0670c81 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:22.965465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a0af7-d071-4f5f-85b3-f0f1c0670c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:22.965739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a0af7-d071-4f5f-85b3-f0f1c0670c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:22.976193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a0af7-d071-4f5f-85b3-f0f1c0670c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:22.977719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a0af7-d071-4f5f-85b3-f0f1c0670c81 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:22.987540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1298/2611] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:23.268163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d930bd0-6dd5-4879-aad1-82d6e8581e1c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:23.269754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0d930bd0-6dd5-4879-aad1-82d6e8581e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:23.282827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d930bd0-6dd5-4879-aad1-82d6e8581e1c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:23.288354 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:23.289540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:23.289893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:23.290230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] PATH_INFO: `/v3/projects/3d0cc629be89418d96cfce30ad5b553f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:23.291462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0cc629be89418d96cfce30ad5b553f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:23.293430 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:23.345969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0cc629be89418d96cfce30ad5b553f', 'request_id': 'req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.355087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0cc629be89418d96cfce30ad5b553f', 'request_id': 'req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.360821 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0cc629be89418d96cfce30ad5b553f', 'request_id': 'req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.371702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] The token cache is being invalidate because project 3d0cc629be89418d96cfce30ad5b553f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:23.371982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.372589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.374140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1314/2612] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:23 2022] DELETE /identity/v3/projects/3d0cc629be89418d96cfce30ad5b553f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:42:23.378390 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a0af7-d071-4f5f-85b3-f0f1c0670c81 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:23.388287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a0af7-d071-4f5f-85b3-f0f1c0670c81 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:23.449165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:23.450089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:23.450389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:23.450663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] PATH_INFO: `/v3/users/02c66b0d13e64e849b725cca11d43e68` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:23.451911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c66b0d13e64e849b725cca11d43e68)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:23.453869 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:23.500857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:23.506916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c66b0d13e64e849b725cca11d43e68'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.522135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c66b0d13e64e849b725cca11d43e68'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.531797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c66b0d13e64e849b725cca11d43e68'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.538684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c66b0d13e64e849b725cca11d43e68'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.539317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7fdc2613-4014-496d-b95e-1ee95dd65b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c66b0d13e64e849b725cca11d43e68'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:23.545570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1299/2613] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:23 2022] DELETE /identity/v3/users/02c66b0d13e64e849b725cca11d43e68 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:42:24.082027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:24.083598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:24.097137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14c38f-ff93-4c6f-a58a-ab65ed008fdd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:24.152194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:24.163134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:24.164014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.164488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.164857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] PATH_INFO: `/v3/projects/4de1b86b43df47cd82f1a97df20f7d69` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.166267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de1b86b43df47cd82f1a97df20f7d69)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:24.168211 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:24.215890 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de1b86b43df47cd82f1a97df20f7d69', 'request_id': 'req-eccb9979-9fac-4f68-b0c5-0a044031f95d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:24.228703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de1b86b43df47cd82f1a97df20f7d69', 'request_id': 'req-eccb9979-9fac-4f68-b0c5-0a044031f95d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:24.234398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de1b86b43df47cd82f1a97df20f7d69', 'request_id': 'req-eccb9979-9fac-4f68-b0c5-0a044031f95d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:24.245967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] The token cache is being invalidate because project 4de1b86b43df47cd82f1a97df20f7d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:24.246252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:24.246893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-eccb9979-9fac-4f68-b0c5-0a044031f95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:24.248470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1315/2614] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:24 2022] DELETE /identity/v3/projects/4de1b86b43df47cd82f1a97df20f7d69 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:24.256412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e74856-8634-486b-905c-eeb24126ea41 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.257177 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e74856-8634-486b-905c-eeb24126ea41 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.257353 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e74856-8634-486b-905c-eeb24126ea41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.283116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-03e74856-8634-486b-905c-eeb24126ea41 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:24.284344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-03e74856-8634-486b-905c-eeb24126ea41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:24.357357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1300/2615] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:24.365379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c078-d0c0-4a1a-abb6-284037066fdf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.365664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c078-d0c0-4a1a-abb6-284037066fdf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.365942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c078-d0c0-4a1a-abb6-284037066fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.385822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4b88c078-d0c0-4a1a-abb6-284037066fdf None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:24.387040 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4b88c078-d0c0-4a1a-abb6-284037066fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:24.395859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1316/2616] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:24.401057 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e74856-8634-486b-905c-eeb24126ea41 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:24.411543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e74856-8634-486b-905c-eeb24126ea41 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:24.417570 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b19dd-3b44-4685-bc32-8b566936c356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:24.418286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b19dd-3b44-4685-bc32-8b566936c356 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.418567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b19dd-3b44-4685-bc32-8b566936c356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.418841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b19dd-3b44-4685-bc32-8b566936c356 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.419575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b19dd-3b44-4685-bc32-8b566936c356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:42:24.419874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b19dd-3b44-4685-bc32-8b566936c356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:24.421511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b19dd-3b44-4685-bc32-8b566936c356 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:24.435814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1301/2617] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:42:24.441755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c078-d0c0-4a1a-abb6-284037066fdf None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:24.453693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c078-d0c0-4a1a-abb6-284037066fdf None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:24.458813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aef2f0-870e-42fd-ae51-3d5ed876bf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:24.459486 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aef2f0-870e-42fd-ae51-3d5ed876bf02 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.459765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aef2f0-870e-42fd-ae51-3d5ed876bf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.460036 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aef2f0-870e-42fd-ae51-3d5ed876bf02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.460765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aef2f0-870e-42fd-ae51-3d5ed876bf02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:24.462779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aef2f0-870e-42fd-ae51-3d5ed876bf02 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:24.498176 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1317/2618] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:24 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:24.505794 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b19dd-3b44-4685-bc32-8b566936c356 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:24.516380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b19dd-3b44-4685-bc32-8b566936c356 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:24.579210 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59e42e-7122-4223-a0d0-2fbda07aa48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:24.579901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59e42e-7122-4223-a0d0-2fbda07aa48c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.580219 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59e42e-7122-4223-a0d0-2fbda07aa48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.580512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59e42e-7122-4223-a0d0-2fbda07aa48c admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.581286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59e42e-7122-4223-a0d0-2fbda07aa48c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:24.583263 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59e42e-7122-4223-a0d0-2fbda07aa48c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:24.610277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1302/2619] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:24 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:24.615632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aef2f0-870e-42fd-ae51-3d5ed876bf02 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:24.629901 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aef2f0-870e-42fd-ae51-3d5ed876bf02 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:24.635793 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c794a5-eca6-4fd3-8a0b-c2ea6dfd38d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:24.636528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c794a5-eca6-4fd3-8a0b-c2ea6dfd38d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.637076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c794a5-eca6-4fd3-8a0b-c2ea6dfd38d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.637352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c794a5-eca6-4fd3-8a0b-c2ea6dfd38d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.638168 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c794a5-eca6-4fd3-8a0b-c2ea6dfd38d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:24.639919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c794a5-eca6-4fd3-8a0b-c2ea6dfd38d2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:24.652293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1318/2620] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:24.656485 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59e42e-7122-4223-a0d0-2fbda07aa48c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:24.670435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59e42e-7122-4223-a0d0-2fbda07aa48c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:24.678189 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435e4b0-b402-491a-81cb-f48be9750ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:24.678903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435e4b0-b402-491a-81cb-f48be9750ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.679185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435e4b0-b402-491a-81cb-f48be9750ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.679463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435e4b0-b402-491a-81cb-f48be9750ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.680248 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435e4b0-b402-491a-81cb-f48be9750ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:24.682082 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435e4b0-b402-491a-81cb-f48be9750ae3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:24.693265 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1303/2621] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:24.697920 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c794a5-eca6-4fd3-8a0b-c2ea6dfd38d2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:24.710498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c794a5-eca6-4fd3-8a0b-c2ea6dfd38d2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:24.716003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9236669-74a1-4813-8dd9-8bf1e9ea37ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:24.717124 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9236669-74a1-4813-8dd9-8bf1e9ea37ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.717269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9236669-74a1-4813-8dd9-8bf1e9ea37ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.717500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9236669-74a1-4813-8dd9-8bf1e9ea37ca admin admin] PATH_INFO: `/v3/projects/0535db54fba945cca5f129ac609b878f/users/7596bd6d6182402492028f8e4a39f2f9/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.734810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9236669-74a1-4813-8dd9-8bf1e9ea37ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0535db54fba945cca5f129ac609b878f, user_id=7596bd6d6182402492028f8e4a39f2f9, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:24.737623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9236669-74a1-4813-8dd9-8bf1e9ea37ca admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:24.758156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1319/2622] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:24 2022] PUT /identity/v3/projects/0535db54fba945cca5f129ac609b878f/users/7596bd6d6182402492028f8e4a39f2f9/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:24.772845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed3e41-ce00-432c-b7b8-1334864d9bd8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:24.773122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed3e41-ce00-432c-b7b8-1334864d9bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:24.773725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed3e41-ce00-432c-b7b8-1334864d9bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:24.808190 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d2ed3e41-ce00-432c-b7b8-1334864d9bd8 None None] MFA Rules not processed for user `7596bd6d6182402492028f8e4a39f2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:24.809461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d2ed3e41-ce00-432c-b7b8-1334864d9bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:24.824920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:24.888474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1304/2623] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:24.895785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9236669-74a1-4813-8dd9-8bf1e9ea37ca admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:24.996969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9236669-74a1-4813-8dd9-8bf1e9ea37ca admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:25.002987 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c982af5-1617-4ee4-b6d5-3a6f1c2c0405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:25.003649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c982af5-1617-4ee4-b6d5-3a6f1c2c0405 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:25.003928 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c982af5-1617-4ee4-b6d5-3a6f1c2c0405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:25.004202 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c982af5-1617-4ee4-b6d5-3a6f1c2c0405 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:25.005576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c982af5-1617-4ee4-b6d5-3a6f1c2c0405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:25.008922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c982af5-1617-4ee4-b6d5-3a6f1c2c0405 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:25.052764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1320/2624] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:24 2022] POST /identity/v3/projects => generated 363 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:25.057172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed3e41-ce00-432c-b7b8-1334864d9bd8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:25.071320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed3e41-ce00-432c-b7b8-1334864d9bd8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:25.140703 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a784c7-d1c6-45f0-9ffc-13ffbfa52f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:25.141417 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a784c7-d1c6-45f0-9ffc-13ffbfa52f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:25.141694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a784c7-d1c6-45f0-9ffc-13ffbfa52f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:25.142069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a784c7-d1c6-45f0-9ffc-13ffbfa52f8f admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:25.142781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a784c7-d1c6-45f0-9ffc-13ffbfa52f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:25.144801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a784c7-d1c6-45f0-9ffc-13ffbfa52f8f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:25.172003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1305/2625] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:25 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:25.177458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c982af5-1617-4ee4-b6d5-3a6f1c2c0405 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:25.186183 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:25.193282 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c982af5-1617-4ee4-b6d5-3a6f1c2c0405 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:25.200954 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c8efd-7498-41fd-81b5-460c153f8cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:25.201863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c8efd-7498-41fd-81b5-460c153f8cce admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:25.202144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c8efd-7498-41fd-81b5-460c153f8cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:25.202512 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c8efd-7498-41fd-81b5-460c153f8cce admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:25.203531 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c8efd-7498-41fd-81b5-460c153f8cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:25.205754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c8efd-7498-41fd-81b5-460c153f8cce admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:25.219323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1321/2626] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:25.224363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a784c7-d1c6-45f0-9ffc-13ffbfa52f8f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:25.239867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a784c7-d1c6-45f0-9ffc-13ffbfa52f8f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:25.247815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305291a-26f9-4c48-a8f4-53999ec241c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:25.248968 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305291a-26f9-4c48-a8f4-53999ec241c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:25.249412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305291a-26f9-4c48-a8f4-53999ec241c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:25.249898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305291a-26f9-4c48-a8f4-53999ec241c6 admin admin] PATH_INFO: `/v3/projects/0d5f765d7d1a4279845cfabcd6b5cd08/users/c8e71cfc8e5a4d148b61d94746951584/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:25.268824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305291a-26f9-4c48-a8f4-53999ec241c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5f765d7d1a4279845cfabcd6b5cd08, user_id=c8e71cfc8e5a4d148b61d94746951584, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:25.271511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305291a-26f9-4c48-a8f4-53999ec241c6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:25.291569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1306/2627] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:25 2022] PUT /identity/v3/projects/0d5f765d7d1a4279845cfabcd6b5cd08/users/c8e71cfc8e5a4d148b61d94746951584/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:25.296817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c8efd-7498-41fd-81b5-460c153f8cce admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:25.315517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c8efd-7498-41fd-81b5-460c153f8cce admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:25.390155 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4afd1-89bf-4b23-8cf6-18467d4d603d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:25.390823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4afd1-89bf-4b23-8cf6-18467d4d603d admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:25.391099 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4afd1-89bf-4b23-8cf6-18467d4d603d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:25.391379 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4afd1-89bf-4b23-8cf6-18467d4d603d admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:25.392102 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4afd1-89bf-4b23-8cf6-18467d4d603d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:25.393759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4afd1-89bf-4b23-8cf6-18467d4d603d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:25.404348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1322/2628] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:25.410374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305291a-26f9-4c48-a8f4-53999ec241c6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:25.421946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305291a-26f9-4c48-a8f4-53999ec241c6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:25.428453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae03b2-3502-4ff4-a7e2-5d22575f9fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:25.429389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae03b2-3502-4ff4-a7e2-5d22575f9fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:25.429673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae03b2-3502-4ff4-a7e2-5d22575f9fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:25.429952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae03b2-3502-4ff4-a7e2-5d22575f9fe7 admin admin] PATH_INFO: `/v3/projects/0d5f765d7d1a4279845cfabcd6b5cd08/users/c8e71cfc8e5a4d148b61d94746951584/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:25.432325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae03b2-3502-4ff4-a7e2-5d22575f9fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5f765d7d1a4279845cfabcd6b5cd08, user_id=c8e71cfc8e5a4d148b61d94746951584, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:25.435020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae03b2-3502-4ff4-a7e2-5d22575f9fe7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:25.446807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1307/2629] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:25 2022] PUT /identity/v3/projects/0d5f765d7d1a4279845cfabcd6b5cd08/users/c8e71cfc8e5a4d148b61d94746951584/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:25.457671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121746-4931-4706-aa50-de9cd7fcc103 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:25.458044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121746-4931-4706-aa50-de9cd7fcc103 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:25.458302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121746-4931-4706-aa50-de9cd7fcc103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:25.494066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8e121746-4931-4706-aa50-de9cd7fcc103 None None] MFA Rules not processed for user `c8e71cfc8e5a4d148b61d94746951584`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:25.495294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8e121746-4931-4706-aa50-de9cd7fcc103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:25.568330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1323/2630] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:25 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:25.581828 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae03b2-3502-4ff4-a7e2-5d22575f9fe7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:25.584517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b1ae03b2-3502-4ff4-a7e2-5d22575f9fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:25.655448 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae03b2-3502-4ff4-a7e2-5d22575f9fe7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:25.660031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa537e45-e61d-4990-8904-6dc18f19834a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:25.660678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa537e45-e61d-4990-8904-6dc18f19834a service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:25.660990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa537e45-e61d-4990-8904-6dc18f19834a service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:25.661262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa537e45-e61d-4990-8904-6dc18f19834a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:25.671011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa537e45-e61d-4990-8904-6dc18f19834a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:25.672442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa537e45-e61d-4990-8904-6dc18f19834a service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:25.682019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1308/2631] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:25 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:25.774304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e121746-4931-4706-aa50-de9cd7fcc103 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:25.785074 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e121746-4931-4706-aa50-de9cd7fcc103 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:25.788696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9afa0-df22-4041-bf84-692d4d30c4bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:25.789374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9afa0-df22-4041-bf84-692d4d30c4bb service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:25.789702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9afa0-df22-4041-bf84-692d4d30c4bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:25.789980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9afa0-df22-4041-bf84-692d4d30c4bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:25.850705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb9afa0-df22-4041-bf84-692d4d30c4bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:25.852123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb9afa0-df22-4041-bf84-692d4d30c4bb service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:25.861801 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1324/2632] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:30.623763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa537e45-e61d-4990-8904-6dc18f19834a service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:30.633649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:30.696761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa537e45-e61d-4990-8904-6dc18f19834a service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:30.703112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:30.704005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:30.704256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:30.704548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] PATH_INFO: `/v3/users/7596bd6d6182402492028f8e4a39f2f9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:30.705899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7596bd6d6182402492028f8e4a39f2f9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:30.707847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:30.766742 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7596bd6d6182402492028f8e4a39f2f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:30.779241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7596bd6d6182402492028f8e4a39f2f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:30.789345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7596bd6d6182402492028f8e4a39f2f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:30.797490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7596bd6d6182402492028f8e4a39f2f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:30.798137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7596bd6d6182402492028f8e4a39f2f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:30.804025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1309/2633] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:30 2022] DELETE /identity/v3/users/7596bd6d6182402492028f8e4a39f2f9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:30.809122 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1863cab-7a4f-4da8-b844-d98bc486531f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:30.809702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1863cab-7a4f-4da8-b844-d98bc486531f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:30.809702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1863cab-7a4f-4da8-b844-d98bc486531f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:30.827269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e1863cab-7a4f-4da8-b844-d98bc486531f None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:30.828465 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e1863cab-7a4f-4da8-b844-d98bc486531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:30.833789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:30.891159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1325/2634] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:30.901394 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:30.903757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:30.983170 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236b8cb-822d-4263-9b0d-7b9460900a0c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:30.989550 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9f515-6bac-4dd9-8aa5-2f00e9d8cd28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:30.990599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9f515-6bac-4dd9-8aa5-2f00e9d8cd28 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:30.991034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9f515-6bac-4dd9-8aa5-2f00e9d8cd28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:30.991318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9f515-6bac-4dd9-8aa5-2f00e9d8cd28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:31.003887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9f515-6bac-4dd9-8aa5-2f00e9d8cd28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:31.005438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9f515-6bac-4dd9-8aa5-2f00e9d8cd28 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:31.017272 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1310/2635] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:31.299387 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1863cab-7a4f-4da8-b844-d98bc486531f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:31.300916 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e1863cab-7a4f-4da8-b844-d98bc486531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:31.313285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1863cab-7a4f-4da8-b844-d98bc486531f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:31.318064 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:31.318935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:31.319220 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:31.319498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] PATH_INFO: `/v3/projects/0535db54fba945cca5f129ac609b878f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:31.320699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0535db54fba945cca5f129ac609b878f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:31.322696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:31.374374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0535db54fba945cca5f129ac609b878f', 'request_id': 'req-678f01e2-1a48-4cef-a6c6-1e779441a628'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.383274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0535db54fba945cca5f129ac609b878f', 'request_id': 'req-678f01e2-1a48-4cef-a6c6-1e779441a628'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.389148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0535db54fba945cca5f129ac609b878f', 'request_id': 'req-678f01e2-1a48-4cef-a6c6-1e779441a628'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.400469 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] The token cache is being invalidate because project 0535db54fba945cca5f129ac609b878f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:31.400643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.401270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.402770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1326/2636] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:31 2022] DELETE /identity/v3/projects/0535db54fba945cca5f129ac609b878f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:31.407109 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9f515-6bac-4dd9-8aa5-2f00e9d8cd28 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:31.417445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9f515-6bac-4dd9-8aa5-2f00e9d8cd28 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:31.449681 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:31.480624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:31.481605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:31.481888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:31.482187 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] PATH_INFO: `/v3/users/c8e71cfc8e5a4d148b61d94746951584` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:31.483479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e71cfc8e5a4d148b61d94746951584)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:31.485493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:31.538445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e71cfc8e5a4d148b61d94746951584'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.551542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e71cfc8e5a4d148b61d94746951584'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.561102 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e71cfc8e5a4d148b61d94746951584'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.570685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e71cfc8e5a4d148b61d94746951584'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.571808 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1043c851-7570-4308-b57f-e0a6999f1a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e71cfc8e5a4d148b61d94746951584'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:31.578589 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1311/2637] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:31 2022] DELETE /identity/v3/users/c8e71cfc8e5a4d148b61d94746951584 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:31.874219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:31.875815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:31.890672 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f01e2-1a48-4cef-a6c6-1e779441a628 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:31.948436 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:31.949363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:31.949634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:31.949904 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] PATH_INFO: `/v3/projects/0d5f765d7d1a4279845cfabcd6b5cd08` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:31.951145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5f765d7d1a4279845cfabcd6b5cd08)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:31.953090 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:31.994299 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:32.005382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5f765d7d1a4279845cfabcd6b5cd08', 'request_id': 'req-36e618db-1aca-44e4-af5c-c493d5ec08d9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.015478 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5f765d7d1a4279845cfabcd6b5cd08', 'request_id': 'req-36e618db-1aca-44e4-af5c-c493d5ec08d9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.021261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5f765d7d1a4279845cfabcd6b5cd08', 'request_id': 'req-36e618db-1aca-44e4-af5c-c493d5ec08d9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.033452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] The token cache is being invalidate because project 0d5f765d7d1a4279845cfabcd6b5cd08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:32.033739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.034517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-36e618db-1aca-44e4-af5c-c493d5ec08d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.036721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1327/2638] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:31 2022] DELETE /identity/v3/projects/0d5f765d7d1a4279845cfabcd6b5cd08 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:32.041879 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146458c6-3847-4488-ab8e-fc8e1aecd04a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.042873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146458c6-3847-4488-ab8e-fc8e1aecd04a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.043165 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146458c6-3847-4488-ab8e-fc8e1aecd04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.065591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-146458c6-3847-4488-ab8e-fc8e1aecd04a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:32.066775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-146458c6-3847-4488-ab8e-fc8e1aecd04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:32.129315 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1312/2639] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:32.138594 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa935b-5274-4be0-85ce-4a2bcd31e321 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.138880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa935b-5274-4be0-85ce-4a2bcd31e321 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.139152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa935b-5274-4be0-85ce-4a2bcd31e321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.158695 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-69aa935b-5274-4be0-85ce-4a2bcd31e321 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:32.159897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-69aa935b-5274-4be0-85ce-4a2bcd31e321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:32.168951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1328/2640] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:32.173238 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146458c6-3847-4488-ab8e-fc8e1aecd04a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:32.183522 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146458c6-3847-4488-ab8e-fc8e1aecd04a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:32.189123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebc6c4-acce-4360-ae25-2b1841a838c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:32.189784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebc6c4-acce-4360-ae25-2b1841a838c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.190061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebc6c4-acce-4360-ae25-2b1841a838c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.190332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebc6c4-acce-4360-ae25-2b1841a838c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.191063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebc6c4-acce-4360-ae25-2b1841a838c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:42:32.191367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebc6c4-acce-4360-ae25-2b1841a838c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:32.192996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebc6c4-acce-4360-ae25-2b1841a838c8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:32.205209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1313/2641] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:42:32.209552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa935b-5274-4be0-85ce-4a2bcd31e321 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:32.219471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa935b-5274-4be0-85ce-4a2bcd31e321 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:32.224967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b77fa2-889f-45f8-9a31-3c54aa46d094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:32.225636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b77fa2-889f-45f8-9a31-3c54aa46d094 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.225938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b77fa2-889f-45f8-9a31-3c54aa46d094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.226215 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b77fa2-889f-45f8-9a31-3c54aa46d094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.226914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b77fa2-889f-45f8-9a31-3c54aa46d094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:32.228841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b77fa2-889f-45f8-9a31-3c54aa46d094 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:32.258671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1329/2642] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:32 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:32.262870 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebc6c4-acce-4360-ae25-2b1841a838c8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:32.273242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebc6c4-acce-4360-ae25-2b1841a838c8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:32.311604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:32.343733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92f1d6-fe96-4b65-9f69-432a3cbec157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:32.344451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92f1d6-fe96-4b65-9f69-432a3cbec157 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.344875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92f1d6-fe96-4b65-9f69-432a3cbec157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.345554 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92f1d6-fe96-4b65-9f69-432a3cbec157 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.346320 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92f1d6-fe96-4b65-9f69-432a3cbec157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:32.348240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92f1d6-fe96-4b65-9f69-432a3cbec157 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:32.372318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1314/2643] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:32 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:32.376556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b77fa2-889f-45f8-9a31-3c54aa46d094 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:32.387591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b77fa2-889f-45f8-9a31-3c54aa46d094 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:32.392603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f043-4fff-4c74-be4c-726bcf8344e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:32.393298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f043-4fff-4c74-be4c-726bcf8344e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.393586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f043-4fff-4c74-be4c-726bcf8344e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.393867 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f043-4fff-4c74-be4c-726bcf8344e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.394601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f043-4fff-4c74-be4c-726bcf8344e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:32.396216 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f043-4fff-4c74-be4c-726bcf8344e4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:32.409395 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1330/2644] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:32.415158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92f1d6-fe96-4b65-9f69-432a3cbec157 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:32.426142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92f1d6-fe96-4b65-9f69-432a3cbec157 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:32.431112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418a73c-0a65-44fe-991a-244f3378bc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:32.431763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418a73c-0a65-44fe-991a-244f3378bc00 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.432039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418a73c-0a65-44fe-991a-244f3378bc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.432308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418a73c-0a65-44fe-991a-244f3378bc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.433453 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418a73c-0a65-44fe-991a-244f3378bc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:32.435030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418a73c-0a65-44fe-991a-244f3378bc00 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:32.445436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1315/2645] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:32.449621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f043-4fff-4c74-be4c-726bcf8344e4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:32.460107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f043-4fff-4c74-be4c-726bcf8344e4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:32.465754 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93df63c-4f5b-475a-b955-11a13cb07804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:32.466667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93df63c-4f5b-475a-b955-11a13cb07804 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.466958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93df63c-4f5b-475a-b955-11a13cb07804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.467238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93df63c-4f5b-475a-b955-11a13cb07804 admin admin] PATH_INFO: `/v3/projects/9940752cf3d14d61a27496b7294c5429/users/8d6fee5c1aa64cf994c89d4555ee3440/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.483743 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93df63c-4f5b-475a-b955-11a13cb07804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9940752cf3d14d61a27496b7294c5429, user_id=8d6fee5c1aa64cf994c89d4555ee3440, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:32.486462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93df63c-4f5b-475a-b955-11a13cb07804 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:32.502119 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1331/2646] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:32 2022] PUT /identity/v3/projects/9940752cf3d14d61a27496b7294c5429/users/8d6fee5c1aa64cf994c89d4555ee3440/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:32.513296 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611a9a1-02ff-4e4b-bd49-1ba77c6f7b96 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.513638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611a9a1-02ff-4e4b-bd49-1ba77c6f7b96 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.514010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611a9a1-02ff-4e4b-bd49-1ba77c6f7b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.542457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1611a9a1-02ff-4e4b-bd49-1ba77c6f7b96 None None] MFA Rules not processed for user `8d6fee5c1aa64cf994c89d4555ee3440`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:32.543854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1611a9a1-02ff-4e4b-bd49-1ba77c6f7b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:32.609694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1316/2647] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:32.621762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93df63c-4f5b-475a-b955-11a13cb07804 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:32.623618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f93df63c-4f5b-475a-b955-11a13cb07804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:32.683031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93df63c-4f5b-475a-b955-11a13cb07804 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:32.687206 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3218c4e-77b5-43c2-b25a-49ccc70674c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:32.687857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3218c4e-77b5-43c2-b25a-49ccc70674c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.688142 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3218c4e-77b5-43c2-b25a-49ccc70674c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.688414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3218c4e-77b5-43c2-b25a-49ccc70674c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.697185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3218c4e-77b5-43c2-b25a-49ccc70674c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:32.698624 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3218c4e-77b5-43c2-b25a-49ccc70674c0 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:32.707544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1332/2648] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:42:32.731512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611a9a1-02ff-4e4b-bd49-1ba77c6f7b96 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:32.795763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611a9a1-02ff-4e4b-bd49-1ba77c6f7b96 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:32.800840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:32.801709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.801988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.802258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] PATH_INFO: `/v3/users/8d6fee5c1aa64cf994c89d4555ee3440` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.803507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d6fee5c1aa64cf994c89d4555ee3440)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:32.805491 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:32.856657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d6fee5c1aa64cf994c89d4555ee3440'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.857812 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:32.871683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d6fee5c1aa64cf994c89d4555ee3440'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.879714 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d6fee5c1aa64cf994c89d4555ee3440'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.886889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d6fee5c1aa64cf994c89d4555ee3440'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.887520 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d6fee5c1aa64cf994c89d4555ee3440'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:32.893460 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1317/2649] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:32 2022] DELETE /identity/v3/users/8d6fee5c1aa64cf994c89d4555ee3440 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:32.898118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4a651-051e-4e9b-a8a7-28b801193f39 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:32.898610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4a651-051e-4e9b-a8a7-28b801193f39 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:32.898716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4a651-051e-4e9b-a8a7-28b801193f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:32.916269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2bc4a651-051e-4e9b-a8a7-28b801193f39 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:32.917499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2bc4a651-051e-4e9b-a8a7-28b801193f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:32.981644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1333/2650] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:42:32.991517 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:32.993176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:33.053343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a0f0c-6f50-4d1c-afbc-32713efaaae8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:33.060357 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b3068-4006-43f3-9364-f1914d3ace0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:33.061335 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b3068-4006-43f3-9364-f1914d3ace0f service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.061690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b3068-4006-43f3-9364-f1914d3ace0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.062065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b3068-4006-43f3-9364-f1914d3ace0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.076642 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b3068-4006-43f3-9364-f1914d3ace0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:33.078114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b3068-4006-43f3-9364-f1914d3ace0f service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:33.088164 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1318/2651] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:33.371403 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4a651-051e-4e9b-a8a7-28b801193f39 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:33.372889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2bc4a651-051e-4e9b-a8a7-28b801193f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:33.382346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:33.386798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4a651-051e-4e9b-a8a7-28b801193f39 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:33.394262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:33.395285 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.397028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.397028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] PATH_INFO: `/v3/projects/9940752cf3d14d61a27496b7294c5429` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.398907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9940752cf3d14d61a27496b7294c5429)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:33.402769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:33.457415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9940752cf3d14d61a27496b7294c5429', 'request_id': 'req-0813614d-67e6-43ba-84d4-fb96a2a6c602'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:33.469517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9940752cf3d14d61a27496b7294c5429', 'request_id': 'req-0813614d-67e6-43ba-84d4-fb96a2a6c602'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:33.475791 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9940752cf3d14d61a27496b7294c5429', 'request_id': 'req-0813614d-67e6-43ba-84d4-fb96a2a6c602'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:33.487252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] The token cache is being invalidate because project 9940752cf3d14d61a27496b7294c5429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:33.487544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:33.488317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0813614d-67e6-43ba-84d4-fb96a2a6c602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:33.489987 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1334/2652] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:33 2022] DELETE /identity/v3/projects/9940752cf3d14d61a27496b7294c5429 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:33.495027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4f93-fc1d-438d-8fec-628970eb72bb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.495318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4f93-fc1d-438d-8fec-628970eb72bb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.495599 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4f93-fc1d-438d-8fec-628970eb72bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.514672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3c1c4f93-fc1d-438d-8fec-628970eb72bb None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:33.515890 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3c1c4f93-fc1d-438d-8fec-628970eb72bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:33.585829 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1319/2653] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:33.594031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444e80-ebcf-4eab-a47d-3b899cb69fab None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.594302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444e80-ebcf-4eab-a47d-3b899cb69fab None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.594580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444e80-ebcf-4eab-a47d-3b899cb69fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.614494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cf444e80-ebcf-4eab-a47d-3b899cb69fab None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:33.615788 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cf444e80-ebcf-4eab-a47d-3b899cb69fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:33.625229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1335/2654] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:33.630119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c4f93-fc1d-438d-8fec-628970eb72bb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:33.640065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c4f93-fc1d-438d-8fec-628970eb72bb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:33.645030 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a287e-81a5-481e-a695-6e37ab3de16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:33.645698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a287e-81a5-481e-a695-6e37ab3de16c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.645986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a287e-81a5-481e-a695-6e37ab3de16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.646269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a287e-81a5-481e-a695-6e37ab3de16c admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.647055 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a287e-81a5-481e-a695-6e37ab3de16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:42:33.647367 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a287e-81a5-481e-a695-6e37ab3de16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:33.649123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a287e-81a5-481e-a695-6e37ab3de16c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:33.662684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1320/2655] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:42:33.667069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444e80-ebcf-4eab-a47d-3b899cb69fab None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:33.677248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444e80-ebcf-4eab-a47d-3b899cb69fab None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:33.681908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c231aa1-29b9-44f9-9a52-38e2620f9f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:33.682563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c231aa1-29b9-44f9-9a52-38e2620f9f2f admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.682852 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c231aa1-29b9-44f9-9a52-38e2620f9f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.683132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c231aa1-29b9-44f9-9a52-38e2620f9f2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.684482 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c231aa1-29b9-44f9-9a52-38e2620f9f2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:33.686452 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c231aa1-29b9-44f9-9a52-38e2620f9f2f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:33.715320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1336/2656] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:33 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:33.720556 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a287e-81a5-481e-a695-6e37ab3de16c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:33.735069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a287e-81a5-481e-a695-6e37ab3de16c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:33.791072 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:33.799993 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d0d94-c68d-4878-bc3c-b090c6dcd99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:33.800807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d0d94-c68d-4878-bc3c-b090c6dcd99f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.801280 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d0d94-c68d-4878-bc3c-b090c6dcd99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.801601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d0d94-c68d-4878-bc3c-b090c6dcd99f admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.802435 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d0d94-c68d-4878-bc3c-b090c6dcd99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:33.804615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d0d94-c68d-4878-bc3c-b090c6dcd99f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:33.826054 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1321/2657] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:33 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:33.830260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c231aa1-29b9-44f9-9a52-38e2620f9f2f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:33.841824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c231aa1-29b9-44f9-9a52-38e2620f9f2f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:33.847441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08b949-3616-44ab-abbb-183bc8607b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:33.848113 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08b949-3616-44ab-abbb-183bc8607b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.848383 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08b949-3616-44ab-abbb-183bc8607b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.848654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08b949-3616-44ab-abbb-183bc8607b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.849414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08b949-3616-44ab-abbb-183bc8607b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:33.851169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08b949-3616-44ab-abbb-183bc8607b13 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:33.861881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1337/2658] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:33.865973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d0d94-c68d-4878-bc3c-b090c6dcd99f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:33.876683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d0d94-c68d-4878-bc3c-b090c6dcd99f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:33.881667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa9d0d-c8da-4af7-afb4-60004e31cc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:33.882338 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa9d0d-c8da-4af7-afb4-60004e31cc50 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.882623 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa9d0d-c8da-4af7-afb4-60004e31cc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.882902 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa9d0d-c8da-4af7-afb4-60004e31cc50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.883672 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa9d0d-c8da-4af7-afb4-60004e31cc50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:33.885317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa9d0d-c8da-4af7-afb4-60004e31cc50 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:33.897568 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1322/2659] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:33.901859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08b949-3616-44ab-abbb-183bc8607b13 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:33.913716 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08b949-3616-44ab-abbb-183bc8607b13 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:33.919126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a939f-eb4a-4181-981b-7a83f7168da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:33.920020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a939f-eb4a-4181-981b-7a83f7168da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.920298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a939f-eb4a-4181-981b-7a83f7168da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.920571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a939f-eb4a-4181-981b-7a83f7168da7 admin admin] PATH_INFO: `/v3/projects/6f55ad0b61564cdc894ac9f253ce3722/users/1ea47de749c2405a94ca00d8899951ee/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.938337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a939f-eb4a-4181-981b-7a83f7168da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f55ad0b61564cdc894ac9f253ce3722, user_id=1ea47de749c2405a94ca00d8899951ee, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:33.941048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a939f-eb4a-4181-981b-7a83f7168da7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:33.957284 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1338/2660] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:33 2022] PUT /identity/v3/projects/6f55ad0b61564cdc894ac9f253ce3722/users/1ea47de749c2405a94ca00d8899951ee/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:33.968625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23a775-c4c3-4e9c-908f-c6356d266ae8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:33.968831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23a775-c4c3-4e9c-908f-c6356d266ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:33.969137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23a775-c4c3-4e9c-908f-c6356d266ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:33.995857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7a23a775-c4c3-4e9c-908f-c6356d266ae8 None None] MFA Rules not processed for user `1ea47de749c2405a94ca00d8899951ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:33.997611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7a23a775-c4c3-4e9c-908f-c6356d266ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:34.068917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1323/2661] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:34.073664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a939f-eb4a-4181-981b-7a83f7168da7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:34.137304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a939f-eb4a-4181-981b-7a83f7168da7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:34.142875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bfcee-63f4-42c7-b20d-b39c31c947ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:34.143525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bfcee-63f4-42c7-b20d-b39c31c947ab admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:34.143815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bfcee-63f4-42c7-b20d-b39c31c947ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:34.144088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bfcee-63f4-42c7-b20d-b39c31c947ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:34.144783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bfcee-63f4-42c7-b20d-b39c31c947ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:34.146872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bfcee-63f4-42c7-b20d-b39c31c947ab admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:34.180563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1339/2662] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:34 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:34.184987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23a775-c4c3-4e9c-908f-c6356d266ae8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:34.196558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23a775-c4c3-4e9c-908f-c6356d266ae8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:34.263161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759b754-8144-4eff-b532-e978f8aca8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:34.264021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759b754-8144-4eff-b532-e978f8aca8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:34.264434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759b754-8144-4eff-b532-e978f8aca8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:34.264871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759b754-8144-4eff-b532-e978f8aca8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:34.266021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759b754-8144-4eff-b532-e978f8aca8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:34.269128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759b754-8144-4eff-b532-e978f8aca8e1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:34.295025 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1324/2663] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:34 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:34.301117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bfcee-63f4-42c7-b20d-b39c31c947ab admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:34.314663 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bfcee-63f4-42c7-b20d-b39c31c947ab admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:34.319484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18044855-cdde-4d36-aebb-2a647eb7bd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:34.320346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18044855-cdde-4d36-aebb-2a647eb7bd84 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:34.320613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18044855-cdde-4d36-aebb-2a647eb7bd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:34.320939 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18044855-cdde-4d36-aebb-2a647eb7bd84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:34.321694 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18044855-cdde-4d36-aebb-2a647eb7bd84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:34.323299 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18044855-cdde-4d36-aebb-2a647eb7bd84 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:34.325693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:34.339719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1340/2664] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:34.345489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759b754-8144-4eff-b532-e978f8aca8e1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:34.356695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759b754-8144-4eff-b532-e978f8aca8e1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:34.362625 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760ec9b-6739-4e2b-ae0b-e4e87f3bb904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:34.363548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760ec9b-6739-4e2b-ae0b-e4e87f3bb904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:34.363840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760ec9b-6739-4e2b-ae0b-e4e87f3bb904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:34.364114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760ec9b-6739-4e2b-ae0b-e4e87f3bb904 admin admin] PATH_INFO: `/v3/projects/55f42883438f48f4a9d80ce83553f2fb/users/9023a5f3941d4254b3cd96e45f043c75/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:34.382085 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760ec9b-6739-4e2b-ae0b-e4e87f3bb904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f42883438f48f4a9d80ce83553f2fb, user_id=9023a5f3941d4254b3cd96e45f043c75, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:34.386936 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760ec9b-6739-4e2b-ae0b-e4e87f3bb904 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:34.405593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1325/2665] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:34 2022] PUT /identity/v3/projects/55f42883438f48f4a9d80ce83553f2fb/users/9023a5f3941d4254b3cd96e45f043c75/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:34.409873 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18044855-cdde-4d36-aebb-2a647eb7bd84 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:34.420544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18044855-cdde-4d36-aebb-2a647eb7bd84 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:34.481863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323a25d-eaea-4975-bdaa-f28b12504d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:34.482544 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323a25d-eaea-4975-bdaa-f28b12504d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:34.482827 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323a25d-eaea-4975-bdaa-f28b12504d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:34.483108 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323a25d-eaea-4975-bdaa-f28b12504d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:34.483841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323a25d-eaea-4975-bdaa-f28b12504d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:34.485508 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323a25d-eaea-4975-bdaa-f28b12504d5a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:34.495283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1341/2666] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:42:34.499697 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760ec9b-6739-4e2b-ae0b-e4e87f3bb904 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:34.510259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760ec9b-6739-4e2b-ae0b-e4e87f3bb904 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:34.515122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f88a6-ad12-4214-8d9b-a6c3e3b3972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:34.515955 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f88a6-ad12-4214-8d9b-a6c3e3b3972d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:34.516234 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f88a6-ad12-4214-8d9b-a6c3e3b3972d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:34.516533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f88a6-ad12-4214-8d9b-a6c3e3b3972d admin admin] PATH_INFO: `/v3/projects/55f42883438f48f4a9d80ce83553f2fb/users/9023a5f3941d4254b3cd96e45f043c75/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:34.518552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f88a6-ad12-4214-8d9b-a6c3e3b3972d admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f42883438f48f4a9d80ce83553f2fb, user_id=9023a5f3941d4254b3cd96e45f043c75, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:34.521233 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f88a6-ad12-4214-8d9b-a6c3e3b3972d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:34.532872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1326/2667] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:34 2022] PUT /identity/v3/projects/55f42883438f48f4a9d80ce83553f2fb/users/9023a5f3941d4254b3cd96e45f043c75/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:34.543406 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27291d-2b01-44fb-ba9e-c39ca78fc5df None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:34.543706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27291d-2b01-44fb-ba9e-c39ca78fc5df None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:34.543988 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27291d-2b01-44fb-ba9e-c39ca78fc5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:34.571410 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8a27291d-2b01-44fb-ba9e-c39ca78fc5df None None] MFA Rules not processed for user `9023a5f3941d4254b3cd96e45f043c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:34.572622 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8a27291d-2b01-44fb-ba9e-c39ca78fc5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:34.647956 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1342/2668] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:42:34.663116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f88a6-ad12-4214-8d9b-a6c3e3b3972d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:34.665114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b23f88a6-ad12-4214-8d9b-a6c3e3b3972d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:34.687478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:34.729079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f88a6-ad12-4214-8d9b-a6c3e3b3972d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:34.733687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d196c5-d673-47fc-b614-0dd49f1edde7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:34.734311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d196c5-d673-47fc-b614-0dd49f1edde7 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:34.734593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d196c5-d673-47fc-b614-0dd49f1edde7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:34.734901 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d196c5-d673-47fc-b614-0dd49f1edde7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:34.808755 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d196c5-d673-47fc-b614-0dd49f1edde7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:34.810240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d196c5-d673-47fc-b614-0dd49f1edde7 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:34.819455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1327/2669] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:37.371598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27291d-2b01-44fb-ba9e-c39ca78fc5df None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:37.438228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27291d-2b01-44fb-ba9e-c39ca78fc5df None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:37.444104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:37.445159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:37.445427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:37.445726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] PATH_INFO: `/v3/users/1ea47de749c2405a94ca00d8899951ee` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:37.447070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea47de749c2405a94ca00d8899951ee)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:37.449045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:37.472536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:37.503152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea47de749c2405a94ca00d8899951ee'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:37.516049 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea47de749c2405a94ca00d8899951ee'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:37.524641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea47de749c2405a94ca00d8899951ee'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:37.525295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea47de749c2405a94ca00d8899951ee'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:37.532386 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea47de749c2405a94ca00d8899951ee'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:37.538568 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1343/2670] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:37 2022] DELETE /identity/v3/users/1ea47de749c2405a94ca00d8899951ee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:37.543287 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8b3a4-787e-4f74-8471-d801c8230e28 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:37.543757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8b3a4-787e-4f74-8471-d801c8230e28 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:37.543953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8b3a4-787e-4f74-8471-d801c8230e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:37.562155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-19e8b3a4-787e-4f74-8471-d801c8230e28 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:37.563576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-19e8b3a4-787e-4f74-8471-d801c8230e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:37.631875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1328/2671] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:37.642851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:37.644323 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:37.706632 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b6ec8-65ff-4734-a49d-f8ab397fc144 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:37.710646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559f9ef-978e-4cdf-88f4-bcee3672510e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:37.711297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559f9ef-978e-4cdf-88f4-bcee3672510e service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:37.711583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559f9ef-978e-4cdf-88f4-bcee3672510e service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:37.711861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559f9ef-978e-4cdf-88f4-bcee3672510e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:37.722226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559f9ef-978e-4cdf-88f4-bcee3672510e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:37.723653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559f9ef-978e-4cdf-88f4-bcee3672510e service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:37.733880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1344/2672] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:38.264237 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8b3a4-787e-4f74-8471-d801c8230e28 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:38.265783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-19e8b3a4-787e-4f74-8471-d801c8230e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:38.278787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8b3a4-787e-4f74-8471-d801c8230e28 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:38.284823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:38.285817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:38.286166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:38.286640 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] PATH_INFO: `/v3/projects/6f55ad0b61564cdc894ac9f253ce3722` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:38.288611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f55ad0b61564cdc894ac9f253ce3722)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:38.313065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:38.367934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f55ad0b61564cdc894ac9f253ce3722', 'request_id': 'req-386bfa63-38ac-4e1a-aff7-795f0665328e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.374212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f55ad0b61564cdc894ac9f253ce3722', 'request_id': 'req-386bfa63-38ac-4e1a-aff7-795f0665328e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.384620 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f55ad0b61564cdc894ac9f253ce3722', 'request_id': 'req-386bfa63-38ac-4e1a-aff7-795f0665328e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.390383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:38.403282 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] The token cache is being invalidate because project 6f55ad0b61564cdc894ac9f253ce3722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:38.403692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.404984 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.406809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1329/2673] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:38 2022] DELETE /identity/v3/projects/6f55ad0b61564cdc894ac9f253ce3722 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:38.410977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559f9ef-978e-4cdf-88f4-bcee3672510e service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:38.421007 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559f9ef-978e-4cdf-88f4-bcee3672510e service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:38.483461 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:38.484363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:38.484646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:38.484971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] PATH_INFO: `/v3/users/9023a5f3941d4254b3cd96e45f043c75` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:38.486169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9023a5f3941d4254b3cd96e45f043c75)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:38.488134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:38.540579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:38.543628 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9023a5f3941d4254b3cd96e45f043c75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.560630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9023a5f3941d4254b3cd96e45f043c75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.571154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9023a5f3941d4254b3cd96e45f043c75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.571457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9023a5f3941d4254b3cd96e45f043c75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.580398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-04073075-a9f3-42ed-ac73-3820396b9b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9023a5f3941d4254b3cd96e45f043c75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:38.586946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1345/2674] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:38 2022] DELETE /identity/v3/users/9023a5f3941d4254b3cd96e45f043c75 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:38.918169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:38.919757 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:38.941694 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bfa63-38ac-4e1a-aff7-795f0665328e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:39.038106 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:39.039379 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.039661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.039933 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] PATH_INFO: `/v3/projects/55f42883438f48f4a9d80ce83553f2fb` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.041224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f42883438f48f4a9d80ce83553f2fb)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:39.043440 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:39.104073 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f42883438f48f4a9d80ce83553f2fb', 'request_id': 'req-b7bd665a-5d4f-403f-a953-c70dc5137dcb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:39.111948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f42883438f48f4a9d80ce83553f2fb', 'request_id': 'req-b7bd665a-5d4f-403f-a953-c70dc5137dcb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:39.123188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f42883438f48f4a9d80ce83553f2fb', 'request_id': 'req-b7bd665a-5d4f-403f-a953-c70dc5137dcb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:39.135847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] The token cache is being invalidate because project 55f42883438f48f4a9d80ce83553f2fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:39.136136 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:39.136823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7bd665a-5d4f-403f-a953-c70dc5137dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:39.138383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1330/2675] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:38 2022] DELETE /identity/v3/projects/55f42883438f48f4a9d80ce83553f2fb => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:39.144280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23aed0-8d13-4da1-87e8-a769f740e8ef None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.144576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23aed0-8d13-4da1-87e8-a769f740e8ef None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.144885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23aed0-8d13-4da1-87e8-a769f740e8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.163441 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-eb23aed0-8d13-4da1-87e8-a769f740e8ef None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:39.164658 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-eb23aed0-8d13-4da1-87e8-a769f740e8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:39.225650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1346/2676] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:39.233849 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6de8f-3d95-4a5f-971a-bd23cc7ea531 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.234138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6de8f-3d95-4a5f-971a-bd23cc7ea531 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.234420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6de8f-3d95-4a5f-971a-bd23cc7ea531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.254132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-82c6de8f-3d95-4a5f-971a-bd23cc7ea531 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:39.255347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-82c6de8f-3d95-4a5f-971a-bd23cc7ea531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:39.264141 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1331/2677] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:39.269270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23aed0-8d13-4da1-87e8-a769f740e8ef None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:39.280388 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23aed0-8d13-4da1-87e8-a769f740e8ef None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:39.285736 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc48d69-8a94-47c6-bd05-639d96cacb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:39.286414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc48d69-8a94-47c6-bd05-639d96cacb4d admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.286703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc48d69-8a94-47c6-bd05-639d96cacb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.287660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc48d69-8a94-47c6-bd05-639d96cacb4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.288779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc48d69-8a94-47c6-bd05-639d96cacb4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:42:39.289044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc48d69-8a94-47c6-bd05-639d96cacb4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:39.290669 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc48d69-8a94-47c6-bd05-639d96cacb4d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:39.304720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1347/2678] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:42:39.309548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6de8f-3d95-4a5f-971a-bd23cc7ea531 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:39.319843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6de8f-3d95-4a5f-971a-bd23cc7ea531 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:39.324843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69257d8f-d633-4332-b9be-9027e05ca90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:39.325345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69257d8f-d633-4332-b9be-9027e05ca90b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.325633 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69257d8f-d633-4332-b9be-9027e05ca90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.326090 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69257d8f-d633-4332-b9be-9027e05ca90b admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.326904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69257d8f-d633-4332-b9be-9027e05ca90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:39.329008 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69257d8f-d633-4332-b9be-9027e05ca90b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:39.364156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1332/2679] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:39 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:39.365569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc48d69-8a94-47c6-bd05-639d96cacb4d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:39.376371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc48d69-8a94-47c6-bd05-639d96cacb4d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:39.436535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c604a93-4841-42cb-b5c5-2263d27e2b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:39.437272 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c604a93-4841-42cb-b5c5-2263d27e2b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.437521 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c604a93-4841-42cb-b5c5-2263d27e2b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.437802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c604a93-4841-42cb-b5c5-2263d27e2b29 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.438975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c604a93-4841-42cb-b5c5-2263d27e2b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:39.440993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c604a93-4841-42cb-b5c5-2263d27e2b29 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:39.464474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1348/2680] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:39 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:39.468597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69257d8f-d633-4332-b9be-9027e05ca90b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:39.480987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69257d8f-d633-4332-b9be-9027e05ca90b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:39.485778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98b243-030e-4947-bdf7-b1798f3e14c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:39.486456 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98b243-030e-4947-bdf7-b1798f3e14c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.486741 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98b243-030e-4947-bdf7-b1798f3e14c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.487019 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98b243-030e-4947-bdf7-b1798f3e14c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.487756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98b243-030e-4947-bdf7-b1798f3e14c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:39.489468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98b243-030e-4947-bdf7-b1798f3e14c0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:39.500443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1333/2681] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:42:39.504425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c604a93-4841-42cb-b5c5-2263d27e2b29 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:39.516327 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c604a93-4841-42cb-b5c5-2263d27e2b29 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:39.521621 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75318601-b742-44d3-849d-443370e63540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:39.522300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75318601-b742-44d3-849d-443370e63540 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.522588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75318601-b742-44d3-849d-443370e63540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.522862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75318601-b742-44d3-849d-443370e63540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.523598 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75318601-b742-44d3-849d-443370e63540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:39.525234 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75318601-b742-44d3-849d-443370e63540 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:39.536322 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1349/2682] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:39.540487 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98b243-030e-4947-bdf7-b1798f3e14c0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:39.551977 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98b243-030e-4947-bdf7-b1798f3e14c0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:39.556609 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3968507-4d4c-4b78-b204-aa186576f68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:39.557577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3968507-4d4c-4b78-b204-aa186576f68d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.557860 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3968507-4d4c-4b78-b204-aa186576f68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.558134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3968507-4d4c-4b78-b204-aa186576f68d admin admin] PATH_INFO: `/v3/projects/edb7c563eb98401bb177967a622c376a/users/5f85a46832fc41cb8407acc668a2b687/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.575789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3968507-4d4c-4b78-b204-aa186576f68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb7c563eb98401bb177967a622c376a, user_id=5f85a46832fc41cb8407acc668a2b687, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:39.578930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3968507-4d4c-4b78-b204-aa186576f68d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:39.598512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1334/2683] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:39 2022] PUT /identity/v3/projects/edb7c563eb98401bb177967a622c376a/users/5f85a46832fc41cb8407acc668a2b687/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:39.609834 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290648d-2ca0-485a-99de-26421d8f617a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.610133 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290648d-2ca0-485a-99de-26421d8f617a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.610417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290648d-2ca0-485a-99de-26421d8f617a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.637804 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4290648d-2ca0-485a-99de-26421d8f617a None None] MFA Rules not processed for user `5f85a46832fc41cb8407acc668a2b687`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:39.639120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4290648d-2ca0-485a-99de-26421d8f617a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:39.652815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:39.709930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1350/2684] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:42:39.721804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3968507-4d4c-4b78-b204-aa186576f68d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:39.724345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b3968507-4d4c-4b78-b204-aa186576f68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:39.748011 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:39.785884 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3968507-4d4c-4b78-b204-aa186576f68d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:39.789521 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0862ccd-6749-46e2-9187-d68893f781c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:39.790178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0862ccd-6749-46e2-9187-d68893f781c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:39.790462 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0862ccd-6749-46e2-9187-d68893f781c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:39.790736 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0862ccd-6749-46e2-9187-d68893f781c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:39.800001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0862ccd-6749-46e2-9187-d68893f781c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:39.801445 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0862ccd-6749-46e2-9187-d68893f781c4 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:39.811309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1335/2685] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:45.080575 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290648d-2ca0-485a-99de-26421d8f617a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:45.082081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4290648d-2ca0-485a-99de-26421d8f617a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:45.141848 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290648d-2ca0-485a-99de-26421d8f617a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:45.146318 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b996fe4-cfd4-44f2-a68f-3d96fce14ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:45.146965 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b996fe4-cfd4-44f2-a68f-3d96fce14ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:45.147252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b996fe4-cfd4-44f2-a68f-3d96fce14ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:45.147527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b996fe4-cfd4-44f2-a68f-3d96fce14ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:45.149429 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8b996fe4-cfd4-44f2-a68f-3d96fce14ef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:45.217829 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b996fe4-cfd4-44f2-a68f-3d96fce14ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:45.219752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b996fe4-cfd4-44f2-a68f-3d96fce14ef1 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:45.230631 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1351/2686] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:48.147805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0862ccd-6749-46e2-9187-d68893f781c4 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:48.149635 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a0862ccd-6749-46e2-9187-d68893f781c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:48.170535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0862ccd-6749-46e2-9187-d68893f781c4 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:48.175932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:48.176997 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:48.177235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:48.177509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] PATH_INFO: `/v3/users/3e4ed08c1c81453abaa6c1dd826f0258` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:48.178847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4ed08c1c81453abaa6c1dd826f0258)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:48.180960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:48.246883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4ed08c1c81453abaa6c1dd826f0258'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.259219 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4ed08c1c81453abaa6c1dd826f0258'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.269958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4ed08c1c81453abaa6c1dd826f0258'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.278455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4ed08c1c81453abaa6c1dd826f0258'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.279163 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e5b39059-8127-445f-a34c-077195396063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4ed08c1c81453abaa6c1dd826f0258'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.286653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1336/2687] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:48 2022] DELETE /identity/v3/users/3e4ed08c1c81453abaa6c1dd826f0258 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:42:48.485014 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b996fe4-cfd4-44f2-a68f-3d96fce14ef1 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:48.495559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b996fe4-cfd4-44f2-a68f-3d96fce14ef1 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:48.555530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:48.556425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:48.556704 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:48.557009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] PATH_INFO: `/v3/projects/281651371cba4d4a9454b0747947cc83` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:48.558208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] RBAC: Authorizing `identity:delete_project(project_id=281651371cba4d4a9454b0747947cc83)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:48.560116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:48.567971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:48.609199 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281651371cba4d4a9454b0747947cc83', 'request_id': 'req-9fe73add-e8c6-4f43-ba26-848f5189d44d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.618680 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281651371cba4d4a9454b0747947cc83', 'request_id': 'req-9fe73add-e8c6-4f43-ba26-848f5189d44d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.624627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281651371cba4d4a9454b0747947cc83', 'request_id': 'req-9fe73add-e8c6-4f43-ba26-848f5189d44d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.635889 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] The token cache is being invalidate because project 281651371cba4d4a9454b0747947cc83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:48.636180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.636860 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9fe73add-e8c6-4f43-ba26-848f5189d44d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:48.638409 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1352/2688] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:48 2022] DELETE /identity/v3/projects/281651371cba4d4a9454b0747947cc83 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:48.643739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63da75-f7c4-4769-8be4-3b8bb3bda0e3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:48.644052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63da75-f7c4-4769-8be4-3b8bb3bda0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:48.644434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63da75-f7c4-4769-8be4-3b8bb3bda0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:48.663529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9d63da75-f7c4-4769-8be4-3b8bb3bda0e3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:48.664730 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9d63da75-f7c4-4769-8be4-3b8bb3bda0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:48.731012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1337/2689] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:48.739297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04717bb8-e910-4863-a701-047655917b8e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:48.739589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04717bb8-e910-4863-a701-047655917b8e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:48.739899 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04717bb8-e910-4863-a701-047655917b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:48.758971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-04717bb8-e910-4863-a701-047655917b8e None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:48.760181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-04717bb8-e910-4863-a701-047655917b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:48.769229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1353/2690] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:48.773630 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63da75-f7c4-4769-8be4-3b8bb3bda0e3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:48.784089 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63da75-f7c4-4769-8be4-3b8bb3bda0e3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:48.790023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5c8c3-4050-4294-bab6-1ba5d4588d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:48.790684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5c8c3-4050-4294-bab6-1ba5d4588d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:48.790971 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5c8c3-4050-4294-bab6-1ba5d4588d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:48.791247 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5c8c3-4050-4294-bab6-1ba5d4588d55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:48.791979 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5c8c3-4050-4294-bab6-1ba5d4588d55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:42:48.792286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5c8c3-4050-4294-bab6-1ba5d4588d55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:48.793930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5c8c3-4050-4294-bab6-1ba5d4588d55 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:48.806113 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1338/2691] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:42:48.810587 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04717bb8-e910-4863-a701-047655917b8e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:48.819978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04717bb8-e910-4863-a701-047655917b8e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:48.824975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c4d84-8000-4aeb-b471-8c838569b2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:48.825627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c4d84-8000-4aeb-b471-8c838569b2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:48.825912 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c4d84-8000-4aeb-b471-8c838569b2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:48.826203 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c4d84-8000-4aeb-b471-8c838569b2c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:48.826886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c4d84-8000-4aeb-b471-8c838569b2c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:48.828871 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c4d84-8000-4aeb-b471-8c838569b2c9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:48.857559 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1354/2692] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:48 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:48.866007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5c8c3-4050-4294-bab6-1ba5d4588d55 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:48.876797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5c8c3-4050-4294-bab6-1ba5d4588d55 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:48.897973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:48.939787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6bde-1ea9-4681-95e1-d0c11a28f8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:48.940475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6bde-1ea9-4681-95e1-d0c11a28f8dd admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:48.940781 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6bde-1ea9-4681-95e1-d0c11a28f8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:48.941061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6bde-1ea9-4681-95e1-d0c11a28f8dd admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:48.942029 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd6bde-1ea9-4681-95e1-d0c11a28f8dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:48.943940 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd6bde-1ea9-4681-95e1-d0c11a28f8dd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:48.968394 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1339/2693] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:48 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:48.973103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c4d84-8000-4aeb-b471-8c838569b2c9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:48.984026 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c4d84-8000-4aeb-b471-8c838569b2c9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:48.988853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4e1c5-f8d0-4c8a-8c9d-c042f2787cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:48.989525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4e1c5-f8d0-4c8a-8c9d-c042f2787cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:48.989810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4e1c5-f8d0-4c8a-8c9d-c042f2787cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:48.990087 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4e1c5-f8d0-4c8a-8c9d-c042f2787cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:48.990824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4e1c5-f8d0-4c8a-8c9d-c042f2787cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:48.992423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4e1c5-f8d0-4c8a-8c9d-c042f2787cd0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:49.003361 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1355/2694] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:49.007591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6bde-1ea9-4681-95e1-d0c11a28f8dd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:49.018830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6bde-1ea9-4681-95e1-d0c11a28f8dd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:49.023783 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6862a-7cd3-48bf-91c5-edd71685dbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:49.024444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6862a-7cd3-48bf-91c5-edd71685dbb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:49.024770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6862a-7cd3-48bf-91c5-edd71685dbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:49.025069 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6862a-7cd3-48bf-91c5-edd71685dbb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:49.025803 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6862a-7cd3-48bf-91c5-edd71685dbb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:49.027378 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6862a-7cd3-48bf-91c5-edd71685dbb3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:49.038064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1340/2695] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:49.042546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4e1c5-f8d0-4c8a-8c9d-c042f2787cd0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:49.053317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4e1c5-f8d0-4c8a-8c9d-c042f2787cd0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:49.059602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0ac92-9361-4f65-bbc8-ec196bf0fbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:49.060497 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0ac92-9361-4f65-bbc8-ec196bf0fbda admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:49.060810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0ac92-9361-4f65-bbc8-ec196bf0fbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:49.061085 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0ac92-9361-4f65-bbc8-ec196bf0fbda admin admin] PATH_INFO: `/v3/projects/e946a4548f114b088ac5073b28e31ba0/users/c38435f90f9a4cceba68faec58e9a5e4/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:49.077641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0ac92-9361-4f65-bbc8-ec196bf0fbda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e946a4548f114b088ac5073b28e31ba0, user_id=c38435f90f9a4cceba68faec58e9a5e4, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:49.080692 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0ac92-9361-4f65-bbc8-ec196bf0fbda admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:49.095127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1356/2696] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:49 2022] PUT /identity/v3/projects/e946a4548f114b088ac5073b28e31ba0/users/c38435f90f9a4cceba68faec58e9a5e4/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:49.101078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f7707-c857-4f8b-85d5-daf9e8f9242d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:49.101374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f7707-c857-4f8b-85d5-daf9e8f9242d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:49.101652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f7707-c857-4f8b-85d5-daf9e8f9242d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:49.120134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7e7f7707-c857-4f8b-85d5-daf9e8f9242d None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:49.121299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7e7f7707-c857-4f8b-85d5-daf9e8f9242d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:49.182954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1341/2697] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:42:49.195679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0ac92-9361-4f65-bbc8-ec196bf0fbda admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:49.198016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7cd0ac92-9361-4f65-bbc8-ec196bf0fbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:49.269366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0ac92-9361-4f65-bbc8-ec196bf0fbda admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:49.273126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071f73f-073f-4ca2-9c71-7e35d100dd00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:49.273787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071f73f-073f-4ca2-9c71-7e35d100dd00 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:49.274224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071f73f-073f-4ca2-9c71-7e35d100dd00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:49.274527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071f73f-073f-4ca2-9c71-7e35d100dd00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:49.284681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071f73f-073f-4ca2-9c71-7e35d100dd00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:49.286232 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071f73f-073f-4ca2-9c71-7e35d100dd00 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:49.296272 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1357/2698] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:53.532590 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82812394-60e4-48e0-ab8f-94129210aecf None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:53.533195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82812394-60e4-48e0-ab8f-94129210aecf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:53.533195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82812394-60e4-48e0-ab8f-94129210aecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:53.560537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-82812394-60e4-48e0-ab8f-94129210aecf None None] MFA Rules not processed for user `c38435f90f9a4cceba68faec58e9a5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:53.561713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-82812394-60e4-48e0-ab8f-94129210aecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:53.672605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1342/2699] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:53.692696 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071f73f-073f-4ca2-9c71-7e35d100dd00 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:53.694782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2071f73f-073f-4ca2-9c71-7e35d100dd00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:53.745738 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:53.789214 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071f73f-073f-4ca2-9c71-7e35d100dd00 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:53.795539 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36293fa5-b0ea-454b-a8bf-74fbf16af9e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:53.796259 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36293fa5-b0ea-454b-a8bf-74fbf16af9e1 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:53.796535 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36293fa5-b0ea-454b-a8bf-74fbf16af9e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:53.796919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36293fa5-b0ea-454b-a8bf-74fbf16af9e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:53.806352 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36293fa5-b0ea-454b-a8bf-74fbf16af9e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:53.807859 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36293fa5-b0ea-454b-a8bf-74fbf16af9e1 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:53.819896 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1358/2700] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:55.237232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484b5eb-b7c2-47ad-86c5-fa9beeb62699 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:55.237750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484b5eb-b7c2-47ad-86c5-fa9beeb62699 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:55.237857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484b5eb-b7c2-47ad-86c5-fa9beeb62699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:55.260257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e484b5eb-b7c2-47ad-86c5-fa9beeb62699 None None] MFA Rules not processed for user `c38435f90f9a4cceba68faec58e9a5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:55.261596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e484b5eb-b7c2-47ad-86c5-fa9beeb62699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:55.269133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1343/2701] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:55.600053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36293fa5-b0ea-454b-a8bf-74fbf16af9e1 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:55.610104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36293fa5-b0ea-454b-a8bf-74fbf16af9e1 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:55.613747 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c10529-5b1c-4e69-981b-7808a0a2e20c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:55.614393 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c10529-5b1c-4e69-981b-7808a0a2e20c service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:55.614681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c10529-5b1c-4e69-981b-7808a0a2e20c service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:55.614958 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c10529-5b1c-4e69-981b-7808a0a2e20c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:55.616776 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d3c10529-5b1c-4e69-981b-7808a0a2e20c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:55.626654 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c10529-5b1c-4e69-981b-7808a0a2e20c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:55.628079 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c10529-5b1c-4e69-981b-7808a0a2e20c service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:55.636363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1359/2702] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:57.746204 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484b5eb-b7c2-47ad-86c5-fa9beeb62699 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:57.747650 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e484b5eb-b7c2-47ad-86c5-fa9beeb62699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:57.761157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484b5eb-b7c2-47ad-86c5-fa9beeb62699 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:57.765576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:57.766499 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:57.766778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:57.767048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] PATH_INFO: `/v3/users/5f85a46832fc41cb8407acc668a2b687` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:57.768242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f85a46832fc41cb8407acc668a2b687)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:57.770182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:57.795492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:57.834016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f85a46832fc41cb8407acc668a2b687'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:57.851558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f85a46832fc41cb8407acc668a2b687'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:57.860092 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f85a46832fc41cb8407acc668a2b687'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:57.867789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f85a46832fc41cb8407acc668a2b687'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:57.868421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f85a46832fc41cb8407acc668a2b687'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:57.876539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1344/2703] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:42:57 2022] DELETE /identity/v3/users/5f85a46832fc41cb8407acc668a2b687 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:57.881209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079295d-3f99-4cbf-9f1a-e8e590d9e8cb None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:57.881742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079295d-3f99-4cbf-9f1a-e8e590d9e8cb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:57.882063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079295d-3f99-4cbf-9f1a-e8e590d9e8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:57.901276 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e079295d-3f99-4cbf-9f1a-e8e590d9e8cb None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:57.902775 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e079295d-3f99-4cbf-9f1a-e8e590d9e8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:57.979764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1360/2704] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:57.992916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:57.994343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:58.052472 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d4fcd-1eda-4f3e-b827-97dc1699c53d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:58.056167 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc9669-2dab-42d0-a803-008cfc55e034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:58.057088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc9669-2dab-42d0-a803-008cfc55e034 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:58.057374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc9669-2dab-42d0-a803-008cfc55e034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:58.057632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc9669-2dab-42d0-a803-008cfc55e034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:58.069872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc9669-2dab-42d0-a803-008cfc55e034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:58.071341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc9669-2dab-42d0-a803-008cfc55e034 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:58.084198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1345/2705] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:42:58.528563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079295d-3f99-4cbf-9f1a-e8e590d9e8cb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:58.530053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e079295d-3f99-4cbf-9f1a-e8e590d9e8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:58.543844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079295d-3f99-4cbf-9f1a-e8e590d9e8cb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:58.558294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:58.559779 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:58.560233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:58.560615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] PATH_INFO: `/v3/projects/edb7c563eb98401bb177967a622c376a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:58.562071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb7c563eb98401bb177967a622c376a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:58.564825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:58.646777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb7c563eb98401bb177967a622c376a', 'request_id': 'req-76692ca9-c696-4c01-843d-485dc88ad68c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:58.656757 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb7c563eb98401bb177967a622c376a', 'request_id': 'req-76692ca9-c696-4c01-843d-485dc88ad68c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:58.663900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb7c563eb98401bb177967a622c376a', 'request_id': 'req-76692ca9-c696-4c01-843d-485dc88ad68c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:58.677751 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] The token cache is being invalidate because project edb7c563eb98401bb177967a622c376a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:42:58.678411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:58.679291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-76692ca9-c696-4c01-843d-485dc88ad68c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:42:58.681279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1361/2706] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:42:58 2022] DELETE /identity/v3/projects/edb7c563eb98401bb177967a622c376a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:58.686110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56979a-881f-45d5-b030-d0d888ce9fbe None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:58.686405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56979a-881f-45d5-b030-d0d888ce9fbe None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:58.686686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56979a-881f-45d5-b030-d0d888ce9fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:58.709739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-af56979a-881f-45d5-b030-d0d888ce9fbe None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:58.710974 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-af56979a-881f-45d5-b030-d0d888ce9fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:58.779708 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1346/2707] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:58.788277 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcf94b-b9ca-4c63-bb24-f448e8d9b7fb None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:58.788572 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcf94b-b9ca-4c63-bb24-f448e8d9b7fb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:58.788904 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcf94b-b9ca-4c63-bb24-f448e8d9b7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:58.807350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-63dcf94b-b9ca-4c63-bb24-f448e8d9b7fb None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:58.808585 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-63dcf94b-b9ca-4c63-bb24-f448e8d9b7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:58.816842 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1362/2708] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:58.821046 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56979a-881f-45d5-b030-d0d888ce9fbe None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:58.830928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56979a-881f-45d5-b030-d0d888ce9fbe None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:58.835535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6f0e1-8827-4206-b651-47d947b2a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:58.836195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6f0e1-8827-4206-b651-47d947b2a898 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:58.836474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6f0e1-8827-4206-b651-47d947b2a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:58.836765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6f0e1-8827-4206-b651-47d947b2a898 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:58.837505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6f0e1-8827-4206-b651-47d947b2a898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:42:58.837809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6f0e1-8827-4206-b651-47d947b2a898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:58.839399 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6f0e1-8827-4206-b651-47d947b2a898 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:58.857557 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1347/2709] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:42:58.858451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcf94b-b9ca-4c63-bb24-f448e8d9b7fb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:58.868293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcf94b-b9ca-4c63-bb24-f448e8d9b7fb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:58.873053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86faa8-a09d-44dd-9d3d-cfb71e7a53ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:58.873713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86faa8-a09d-44dd-9d3d-cfb71e7a53ff admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:58.873994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86faa8-a09d-44dd-9d3d-cfb71e7a53ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:58.874270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86faa8-a09d-44dd-9d3d-cfb71e7a53ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:58.875264 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86faa8-a09d-44dd-9d3d-cfb71e7a53ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:58.877413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86faa8-a09d-44dd-9d3d-cfb71e7a53ff admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:58.905484 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1363/2710] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:42:58 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:42:58.910181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6f0e1-8827-4206-b651-47d947b2a898 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:58.920899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6f0e1-8827-4206-b651-47d947b2a898 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:58.945277 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:58.980324 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52366ff2-324d-4e5f-b9ee-b171ee9f83d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:58.981042 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52366ff2-324d-4e5f-b9ee-b171ee9f83d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:58.981328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52366ff2-324d-4e5f-b9ee-b171ee9f83d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:58.981606 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52366ff2-324d-4e5f-b9ee-b171ee9f83d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:58.982317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52366ff2-324d-4e5f-b9ee-b171ee9f83d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:58.984340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52366ff2-324d-4e5f-b9ee-b171ee9f83d4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:59.008551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1348/2711] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:42:58 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:42:59.010015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86faa8-a09d-44dd-9d3d-cfb71e7a53ff admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:59.015479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:42:59.022738 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86faa8-a09d-44dd-9d3d-cfb71e7a53ff admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:59.030224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d167e8-c104-4e69-8ac5-a8538c2f806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:59.030990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d167e8-c104-4e69-8ac5-a8538c2f806b admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:59.031330 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d167e8-c104-4e69-8ac5-a8538c2f806b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:59.031611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d167e8-c104-4e69-8ac5-a8538c2f806b admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:59.032342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d167e8-c104-4e69-8ac5-a8538c2f806b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:59.033991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d167e8-c104-4e69-8ac5-a8538c2f806b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:59.046230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1364/2712] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:59.050381 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52366ff2-324d-4e5f-b9ee-b171ee9f83d4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:59.062071 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52366ff2-324d-4e5f-b9ee-b171ee9f83d4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:59.067804 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85c6d9-93bf-4505-a4e0-ac304c99fd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:59.068538 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85c6d9-93bf-4505-a4e0-ac304c99fd87 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:59.068845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85c6d9-93bf-4505-a4e0-ac304c99fd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:59.069121 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85c6d9-93bf-4505-a4e0-ac304c99fd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:59.069857 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85c6d9-93bf-4505-a4e0-ac304c99fd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:59.071474 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85c6d9-93bf-4505-a4e0-ac304c99fd87 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:59.084610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1349/2713] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:42:59.088991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d167e8-c104-4e69-8ac5-a8538c2f806b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:59.099837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d167e8-c104-4e69-8ac5-a8538c2f806b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:59.104446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea83e2-8499-4bcf-b6b2-59ce7116dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:59.105464 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea83e2-8499-4bcf-b6b2-59ce7116dd23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:59.105743 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea83e2-8499-4bcf-b6b2-59ce7116dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:59.106015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea83e2-8499-4bcf-b6b2-59ce7116dd23 admin admin] PATH_INFO: `/v3/projects/c2ca2c1ee6294b2fb8020ea0b1d73d72/users/08dfca461f9947c08f885161891deb10/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:59.121019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea83e2-8499-4bcf-b6b2-59ce7116dd23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ca2c1ee6294b2fb8020ea0b1d73d72, user_id=08dfca461f9947c08f885161891deb10, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:59.123662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea83e2-8499-4bcf-b6b2-59ce7116dd23 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:59.138255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1365/2714] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:42:59 2022] PUT /identity/v3/projects/c2ca2c1ee6294b2fb8020ea0b1d73d72/users/08dfca461f9947c08f885161891deb10/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:42:59.143616 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651fd7-43c4-4fca-9ea6-87ddc1b43ef8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:59.143896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651fd7-43c4-4fca-9ea6-87ddc1b43ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:59.144194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651fd7-43c4-4fca-9ea6-87ddc1b43ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:59.165505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c6651fd7-43c4-4fca-9ea6-87ddc1b43ef8 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:42:59.166815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c6651fd7-43c4-4fca-9ea6-87ddc1b43ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:59.229922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1350/2715] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:42:59.240421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea83e2-8499-4bcf-b6b2-59ce7116dd23 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:42:59.242365 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-74ea83e2-8499-4bcf-b6b2-59ce7116dd23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:42:59.301815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea83e2-8499-4bcf-b6b2-59ce7116dd23 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:42:59.306764 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdfc97-9387-4c13-9e06-57594452f4c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:42:59.307438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdfc97-9387-4c13-9e06-57594452f4c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:42:59.307719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdfc97-9387-4c13-9e06-57594452f4c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:42:59.308006 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdfc97-9387-4c13-9e06-57594452f4c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:42:59.318566 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdfc97-9387-4c13-9e06-57594452f4c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:42:59.319996 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdfc97-9387-4c13-9e06-57594452f4c5 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:42:59.330715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1366/2716] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:43:04.264385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27a5b4-475c-4a67-a914-b21a6f23038a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:04.264917 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27a5b4-475c-4a67-a914-b21a6f23038a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:04.269850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27a5b4-475c-4a67-a914-b21a6f23038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:04.303209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4b27a5b4-475c-4a67-a914-b21a6f23038a None None] MFA Rules not processed for user `08dfca461f9947c08f885161891deb10`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:43:04.304436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4b27a5b4-475c-4a67-a914-b21a6f23038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:04.377978 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1351/2717] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:43:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:43:04.379227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdfc97-9387-4c13-9e06-57594452f4c5 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:04.390209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdfc97-9387-4c13-9e06-57594452f4c5 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:04.399919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c2791-02a5-4bb6-97d5-0de2e40f0a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:04.400399 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c2791-02a5-4bb6-97d5-0de2e40f0a33 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:04.400684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c2791-02a5-4bb6-97d5-0de2e40f0a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:04.401000 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c2791-02a5-4bb6-97d5-0de2e40f0a33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:04.403148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c2791-02a5-4bb6-97d5-0de2e40f0a33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:04.403814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c2791-02a5-4bb6-97d5-0de2e40f0a33 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:04.438975 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1367/2718] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:43:04 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:43:04.447709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27a5b4-475c-4a67-a914-b21a6f23038a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:04.465336 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27a5b4-475c-4a67-a914-b21a6f23038a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:04.505023 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:43:04.540684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9faebb-ac28-43a9-b6c4-099961ec0a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:04.542347 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9faebb-ac28-43a9-b6c4-099961ec0a61 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:04.542833 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9faebb-ac28-43a9-b6c4-099961ec0a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:04.543119 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9faebb-ac28-43a9-b6c4-099961ec0a61 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:04.544007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9faebb-ac28-43a9-b6c4-099961ec0a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:04.546950 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9faebb-ac28-43a9-b6c4-099961ec0a61 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:04.571921 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1352/2719] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:43:04 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:43:04.575684 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c2791-02a5-4bb6-97d5-0de2e40f0a33 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:04.586735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c2791-02a5-4bb6-97d5-0de2e40f0a33 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:04.591612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c39c24-b830-4102-b08c-06133e5305d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:04.592226 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c39c24-b830-4102-b08c-06133e5305d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:04.592516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c39c24-b830-4102-b08c-06133e5305d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:04.592818 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c39c24-b830-4102-b08c-06133e5305d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:04.593546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c39c24-b830-4102-b08c-06133e5305d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:04.595261 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c39c24-b830-4102-b08c-06133e5305d9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:04.606930 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1368/2720] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:43:04.611242 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9faebb-ac28-43a9-b6c4-099961ec0a61 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:04.623601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9faebb-ac28-43a9-b6c4-099961ec0a61 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:04.628839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749227a5-568c-4b04-ad2a-bee307a2242b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:04.629852 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749227a5-568c-4b04-ad2a-bee307a2242b admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:04.630132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749227a5-568c-4b04-ad2a-bee307a2242b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:04.630412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749227a5-568c-4b04-ad2a-bee307a2242b admin admin] PATH_INFO: `/v3/projects/cb4549ef556d44c5bc529761d3cd95d3/users/2761b6eb4bc647b18b1da41d186c9502/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:04.650070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749227a5-568c-4b04-ad2a-bee307a2242b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4549ef556d44c5bc529761d3cd95d3, user_id=2761b6eb4bc647b18b1da41d186c9502, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:04.652805 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749227a5-568c-4b04-ad2a-bee307a2242b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:04.671505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1353/2721] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:43:04 2022] PUT /identity/v3/projects/cb4549ef556d44c5bc529761d3cd95d3/users/2761b6eb4bc647b18b1da41d186c9502/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:43:04.672861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c39c24-b830-4102-b08c-06133e5305d9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:04.683556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c39c24-b830-4102-b08c-06133e5305d9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:04.759288 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6346d-fce6-4e03-b1d7-888923797a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:04.759972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6346d-fce6-4e03-b1d7-888923797a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:04.760254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6346d-fce6-4e03-b1d7-888923797a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:04.760533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6346d-fce6-4e03-b1d7-888923797a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:04.761340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6346d-fce6-4e03-b1d7-888923797a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:04.762964 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6346d-fce6-4e03-b1d7-888923797a3a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:04.773999 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1369/2722] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:43:04.778215 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749227a5-568c-4b04-ad2a-bee307a2242b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:04.789361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749227a5-568c-4b04-ad2a-bee307a2242b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:04.794061 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcafe85-02b2-473a-962d-afcb79b2bc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:04.794963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcafe85-02b2-473a-962d-afcb79b2bc22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:04.795245 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcafe85-02b2-473a-962d-afcb79b2bc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:04.795518 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcafe85-02b2-473a-962d-afcb79b2bc22 admin admin] PATH_INFO: `/v3/projects/cb4549ef556d44c5bc529761d3cd95d3/users/2761b6eb4bc647b18b1da41d186c9502/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:04.797601 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcafe85-02b2-473a-962d-afcb79b2bc22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4549ef556d44c5bc529761d3cd95d3, user_id=2761b6eb4bc647b18b1da41d186c9502, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:04.800784 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcafe85-02b2-473a-962d-afcb79b2bc22 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:04.812649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1354/2723] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:43:04 2022] PUT /identity/v3/projects/cb4549ef556d44c5bc529761d3cd95d3/users/2761b6eb4bc647b18b1da41d186c9502/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:43:09.769019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe335f99-3692-461e-af92-318dd25515ea None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:09.769941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe335f99-3692-461e-af92-318dd25515ea None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:09.770474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe335f99-3692-461e-af92-318dd25515ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:09.805634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fe335f99-3692-461e-af92-318dd25515ea None None] MFA Rules not processed for user `2761b6eb4bc647b18b1da41d186c9502`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:43:09.807342 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fe335f99-3692-461e-af92-318dd25515ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:09.831812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:43:09.881634 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1370/2724] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:43:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:43:09.893716 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcafe85-02b2-473a-962d-afcb79b2bc22 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:09.895438 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4dcafe85-02b2-473a-962d-afcb79b2bc22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:09.956259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcafe85-02b2-473a-962d-afcb79b2bc22 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:09.960348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2309a-31d9-4f8a-968a-bc56780ce55d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b75a27126c9464989cec70e5104824c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:09.960975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2309a-31d9-4f8a-968a-bc56780ce55d service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:09.961258 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2309a-31d9-4f8a-968a-bc56780ce55d service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:09.961526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2309a-31d9-4f8a-968a-bc56780ce55d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:10.020976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2309a-31d9-4f8a-968a-bc56780ce55d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:10.022411 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2309a-31d9-4f8a-968a-bc56780ce55d service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:10.031680 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1355/2725] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:43:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:43:10.144317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b82388-ee21-4f5f-9847-2d171820a6ef None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:10.144643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b82388-ee21-4f5f-9847-2d171820a6ef None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:10.144970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b82388-ee21-4f5f-9847-2d171820a6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:10.164944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-06b82388-ee21-4f5f-9847-2d171820a6ef None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:43:10.166157 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-06b82388-ee21-4f5f-9847-2d171820a6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:10.233296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1371/2726] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:43:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:43:10.288478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2309a-31d9-4f8a-968a-bc56780ce55d service glance] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:10.290155 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-37c2309a-31d9-4f8a-968a-bc56780ce55d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:10.345285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:43:10.375581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2309a-31d9-4f8a-968a-bc56780ce55d service glance] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:10.380779 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63b59b-355d-42db-a43d-69adf82b403a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:10.381468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63b59b-355d-42db-a43d-69adf82b403a service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:10.381753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63b59b-355d-42db-a43d-69adf82b403a service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:10.382053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63b59b-355d-42db-a43d-69adf82b403a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:10.393173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63b59b-355d-42db-a43d-69adf82b403a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:10.394801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63b59b-355d-42db-a43d-69adf82b403a service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:10.404477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1356/2727] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:43:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:43:19.026428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2bc92-aa5a-441c-898a-27358d3a50d6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:19.027230 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2bc92-aa5a-441c-898a-27358d3a50d6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:19.027414 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2bc92-aa5a-441c-898a-27358d3a50d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:19.048298 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f4d2bc92-aa5a-441c-898a-27358d3a50d6 None None] MFA Rules not processed for user `08dfca461f9947c08f885161891deb10`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:43:19.049546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f4d2bc92-aa5a-441c-898a-27358d3a50d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:19.058035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1372/2728] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:43:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:43:20.400701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=13 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:43:20.901988 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6eae9-4648-47a7-80d5-39f480e56b02 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:20.902380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6eae9-4648-47a7-80d5-39f480e56b02 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:20.902756 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6eae9-4648-47a7-80d5-39f480e56b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:20.921927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1ec6eae9-4648-47a7-80d5-39f480e56b02 None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:43:20.923228 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1ec6eae9-4648-47a7-80d5-39f480e56b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:20.930870 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1357/2729] 10.222.0.69 () {58 vars in 1009 bytes} [Fri Jun 24 10:43:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:43:20.943477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2bc92-aa5a-441c-898a-27358d3a50d6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:20.956281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2bc92-aa5a-441c-898a-27358d3a50d6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:20.961529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc5ddf-705a-40c4-b002-d0e2f4f233ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:20.962180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc5ddf-705a-40c4-b002-d0e2f4f233ec service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:20.962463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc5ddf-705a-40c4-b002-d0e2f4f233ec service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:20.962739 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc5ddf-705a-40c4-b002-d0e2f4f233ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:20.972145 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc5ddf-705a-40c4-b002-d0e2f4f233ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:20.974008 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc5ddf-705a-40c4-b002-d0e2f4f233ec service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:20.982994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1373/2730] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:43:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:43:30.917667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:43:30.979713 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=53 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:43:33.556340 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6eae9-4648-47a7-80d5-39f480e56b02 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:33.556907 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1ec6eae9-4648-47a7-80d5-39f480e56b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:33.631863 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6eae9-4648-47a7-80d5-39f480e56b02 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:33.637545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72440b8f-2389-4590-af32-bac29bab3112 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:33.638212 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72440b8f-2389-4590-af32-bac29bab3112 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:33.638493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72440b8f-2389-4590-af32-bac29bab3112 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:33.638775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72440b8f-2389-4590-af32-bac29bab3112 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:33.640661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-72440b8f-2389-4590-af32-bac29bab3112 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:33.652743 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72440b8f-2389-4590-af32-bac29bab3112 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:33.654216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72440b8f-2389-4590-af32-bac29bab3112 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:33.664117 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1358/2731] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:43:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:43:43.659831 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=32 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:43:46.819997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc5ddf-705a-40c4-b002-d0e2f4f233ec service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:43:46.879565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc5ddf-705a-40c4-b002-d0e2f4f233ec service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:43:46.883897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ea78e-6ff9-4c7c-b397-0480f98c6063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:43:46.884541 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ea78e-6ff9-4c7c-b397-0480f98c6063 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:46.884861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ea78e-6ff9-4c7c-b397-0480f98c6063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:46.885138 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ea78e-6ff9-4c7c-b397-0480f98c6063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:46.895421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ea78e-6ff9-4c7c-b397-0480f98c6063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:43:46.896897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ea78e-6ff9-4c7c-b397-0480f98c6063 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:43:46.907050 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1374/2732] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:43:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:43:54.810690 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c417e-4814-4c88-b849-214086931386 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:43:54.811213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c417e-4814-4c88-b849-214086931386 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:43:54.811314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c417e-4814-4c88-b849-214086931386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:43:54.829209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-df9c417e-4814-4c88-b849-214086931386 None None] MFA Rules not processed for user `c38435f90f9a4cceba68faec58e9a5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:43:54.830401 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-df9c417e-4814-4c88-b849-214086931386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:43:54.893537 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1359/2733] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:43:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:43:56.902502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=26 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:44:04.538787 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ea78e-6ff9-4c7c-b397-0480f98c6063 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:44:04.541413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3d0ea78e-6ff9-4c7c-b397-0480f98c6063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:44:04.605630 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ea78e-6ff9-4c7c-b397-0480f98c6063 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:44:04.610708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d75bf-6e32-40ef-9688-46ecc3084fcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:44:04.611705 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d75bf-6e32-40ef-9688-46ecc3084fcb service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:44:04.612063 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d75bf-6e32-40ef-9688-46ecc3084fcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:44:04.612420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d75bf-6e32-40ef-9688-46ecc3084fcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:44:04.615645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d36d75bf-6e32-40ef-9688-46ecc3084fcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:44:04.692518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d75bf-6e32-40ef-9688-46ecc3084fcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:44:04.694610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d75bf-6e32-40ef-9688-46ecc3084fcb service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:44:04.707314 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1375/2734] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:44:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:44:04.885688 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=25 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:44:11.654236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937950-4d36-464b-9f08-63f18cfeab6f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:44:11.655017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937950-4d36-464b-9f08-63f18cfeab6f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:44:11.655017 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937950-4d36-464b-9f08-63f18cfeab6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:44:11.675737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-08937950-4d36-464b-9f08-63f18cfeab6f None None] MFA Rules not processed for user `5fc073d480884782b4084952345382b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:44:11.676966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-08937950-4d36-464b-9f08-63f18cfeab6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:44:11.685026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1360/2735] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:44:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:44:14.702300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=48 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:44:21.668291 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:44:57.208554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b171fbd-f8f5-4394-a9be-6b0bdda43a2c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:44:57.209118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b171fbd-f8f5-4394-a9be-6b0bdda43a2c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:44:57.209118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b171fbd-f8f5-4394-a9be-6b0bdda43a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:44:57.226851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3b171fbd-f8f5-4394-a9be-6b0bdda43a2c None None] MFA Rules not processed for user `c38435f90f9a4cceba68faec58e9a5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:44:57.228178 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3b171fbd-f8f5-4394-a9be-6b0bdda43a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:44:57.235557 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1376/2736] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:44:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:45:07.221637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:45:20.496605 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebad4f8-c273-4184-955c-c742dc1c8afa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:20.497197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebad4f8-c273-4184-955c-c742dc1c8afa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:20.497197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebad4f8-c273-4184-955c-c742dc1c8afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:20.514998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5ebad4f8-c273-4184-955c-c742dc1c8afa None None] MFA Rules not processed for user `f2360d67304049278fe17c8abfe44a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:45:20.516188 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5ebad4f8-c273-4184-955c-c742dc1c8afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:20.523607 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1361/2737] 10.222.0.69 () {58 vars in 967 bytes} [Fri Jun 24 10:45:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:45:20.562991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b171fbd-f8f5-4394-a9be-6b0bdda43a2c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:45:20.579415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b171fbd-f8f5-4394-a9be-6b0bdda43a2c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:45:20.589370 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e877991-8dd4-4395-9c7e-0f24518e32b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d5bd3f9dfe45ecb8d1a51d248f3c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:45:20.590021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e877991-8dd4-4395-9c7e-0f24518e32b5 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:20.590280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e877991-8dd4-4395-9c7e-0f24518e32b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:20.590588 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e877991-8dd4-4395-9c7e-0f24518e32b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:20.600799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e877991-8dd4-4395-9c7e-0f24518e32b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:45:20.602446 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e877991-8dd4-4395-9c7e-0f24518e32b5 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:45:20.611450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1377/2738] 10.222.0.69 () {60 vars in 1384 bytes} [Fri Jun 24 10:45:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:45:24.067960 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebad4f8-c273-4184-955c-c742dc1c8afa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:45:24.069840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5ebad4f8-c273-4184-955c-c742dc1c8afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:24.086643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebad4f8-c273-4184-955c-c742dc1c8afa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:45:24.092366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78af8-648b-409d-b6f3-d42cd87c0e05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:45:24.093205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78af8-648b-409d-b6f3-d42cd87c0e05 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:24.093548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78af8-648b-409d-b6f3-d42cd87c0e05 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:24.093951 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78af8-648b-409d-b6f3-d42cd87c0e05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:24.095837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2af78af8-648b-409d-b6f3-d42cd87c0e05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:24.110153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af78af8-648b-409d-b6f3-d42cd87c0e05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:45:24.112138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af78af8-648b-409d-b6f3-d42cd87c0e05 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:45:24.138647 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1362/2739] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:45:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:45:27.057523 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e877991-8dd4-4395-9c7e-0f24518e32b5 service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:45:27.120641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e877991-8dd4-4395-9c7e-0f24518e32b5 service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:45:27.125967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:45:27.126857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:27.127140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:27.127417 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] PATH_INFO: `/v3/users/08dfca461f9947c08f885161891deb10` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:27.128783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] RBAC: Authorizing `identity:delete_user(user_id=08dfca461f9947c08f885161891deb10)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:45:27.130772 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:45:27.182353 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08dfca461f9947c08f885161891deb10'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.194773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08dfca461f9947c08f885161891deb10'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.203870 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08dfca461f9947c08f885161891deb10'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.204480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08dfca461f9947c08f885161891deb10'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.211679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08dfca461f9947c08f885161891deb10'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.218148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1378/2740] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:45:27 2022] DELETE /identity/v3/users/08dfca461f9947c08f885161891deb10 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:45:27.410963 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78af8-648b-409d-b6f3-d42cd87c0e05 service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:45:27.421846 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78af8-648b-409d-b6f3-d42cd87c0e05 service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:45:27.501001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:45:27.501922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:27.502216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:27.502489 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] PATH_INFO: `/v3/projects/c2ca2c1ee6294b2fb8020ea0b1d73d72` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:27.503806 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ca2c1ee6294b2fb8020ea0b1d73d72)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:45:27.506539 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:45:27.578420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ca2c1ee6294b2fb8020ea0b1d73d72', 'request_id': 'req-d96454eb-f9fa-47c5-9714-a7a154405a9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.585466 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ca2c1ee6294b2fb8020ea0b1d73d72', 'request_id': 'req-d96454eb-f9fa-47c5-9714-a7a154405a9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.595664 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ca2c1ee6294b2fb8020ea0b1d73d72', 'request_id': 'req-d96454eb-f9fa-47c5-9714-a7a154405a9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.607610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] The token cache is being invalidate because project c2ca2c1ee6294b2fb8020ea0b1d73d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:45:27.607895 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.608585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.611332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1363/2741] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:45:27 2022] DELETE /identity/v3/projects/c2ca2c1ee6294b2fb8020ea0b1d73d72 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:45:27.615530 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:45:27.617158 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:27.637182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12323886-6ea5-429c-8b2b-96bf48b9e40e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:45:27.714639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:45:27.715608 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:27.716066 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:27.716348 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] PATH_INFO: `/v3/users/2761b6eb4bc647b18b1da41d186c9502` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:27.718238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2761b6eb4bc647b18b1da41d186c9502)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:45:27.720915 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:45:27.729120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:45:27.781953 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2761b6eb4bc647b18b1da41d186c9502'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.796932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2761b6eb4bc647b18b1da41d186c9502'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.811227 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2761b6eb4bc647b18b1da41d186c9502'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.811922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2761b6eb4bc647b18b1da41d186c9502'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.819947 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2761b6eb4bc647b18b1da41d186c9502'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:27.829877 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1379/2742] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:45:27 2022] DELETE /identity/v3/users/2761b6eb4bc647b18b1da41d186c9502 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:45:28.034731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:45:28.036235 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:28.049739 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96454eb-f9fa-47c5-9714-a7a154405a9b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:45:28.122765 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:45:28.123746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:28.124129 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:28.124503 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] PATH_INFO: `/v3/projects/cb4549ef556d44c5bc529761d3cd95d3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:28.126009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4549ef556d44c5bc529761d3cd95d3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:45:28.128146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:45:28.142181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:45:28.186530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4549ef556d44c5bc529761d3cd95d3', 'request_id': 'req-01dcbfdf-2516-4728-bbdf-86b15f674a37'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:28.193698 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4549ef556d44c5bc529761d3cd95d3', 'request_id': 'req-01dcbfdf-2516-4728-bbdf-86b15f674a37'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:28.202330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4549ef556d44c5bc529761d3cd95d3', 'request_id': 'req-01dcbfdf-2516-4728-bbdf-86b15f674a37'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:28.218185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] The token cache is being invalidate because project cb4549ef556d44c5bc529761d3cd95d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:45:28.218661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:28.219643 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:45:28.222091 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1364/2743] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:45:28 2022] DELETE /identity/v3/projects/cb4549ef556d44c5bc529761d3cd95d3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:45:37.822893 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:45:38.213118 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:45:44.506922 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:45:44.508527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:44.574810 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559b64d-6781-4db7-bae0-da983e84c2cd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:45:44.624279 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3f308-9d32-4f1b-8544-5341787fbc43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:45:44.624936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3f308-9d32-4f1b-8544-5341787fbc43 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:44.625222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3f308-9d32-4f1b-8544-5341787fbc43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:44.625500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3f308-9d32-4f1b-8544-5341787fbc43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:44.627435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c8b3f308-9d32-4f1b-8544-5341787fbc43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:44.702477 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3f308-9d32-4f1b-8544-5341787fbc43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:45:44.703942 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3f308-9d32-4f1b-8544-5341787fbc43 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:45:44.715222 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1380/2744] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:45:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:45:51.469737 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:45:51.471236 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:51.485345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcbfdf-2516-4728-bbdf-86b15f674a37 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:45:51.489123 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:45:51.489772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:51.490056 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:51.490333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:51.492154 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:51.505872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:45:51.507348 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:45:51.518898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1365/2745] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:45:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:45:54.709969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=74 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:45:58.560924 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b5e0a-9adb-4cb0-96bd-6ce94166c3d0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:45:58.561571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b5e0a-9adb-4cb0-96bd-6ce94166c3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:45:58.561571 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b5e0a-9adb-4cb0-96bd-6ce94166c3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:45:58.579419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5f3b5e0a-9adb-4cb0-96bd-6ce94166c3d0 None None] MFA Rules not processed for user `c38435f90f9a4cceba68faec58e9a5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:45:58.580649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5f3b5e0a-9adb-4cb0-96bd-6ce94166c3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:45:58.643783 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1381/2746] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:45:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:46:01.513875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:46:08.635687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=25 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:46:20.980176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:46:20.981928 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:46:21.040810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df459f-eb93-4066-af38-5d0ff4f0eca1 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:46:21.044836 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:46:21.045555 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:46:21.045837 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:46:21.046197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:46:21.048441 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:46:21.121817 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:46:21.123256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:46:21.133124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1366/2747] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:46:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:46:27.299022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b5e0a-9adb-4cb0-96bd-6ce94166c3d0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:46:27.300591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5f3b5e0a-9adb-4cb0-96bd-6ce94166c3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:46:27.353809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:46:27.355291 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:46:27.368312 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206adae1-a909-431c-828f-4c896197a955 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:46:27.373009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:46:27.373891 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:46:27.374178 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:46:27.374451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] PATH_INFO: `/v3/users/5fc073d480884782b4084952345382b9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:46:27.375644 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fc073d480884782b4084952345382b9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:46:27.377700 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:46:27.379035 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b5e0a-9adb-4cb0-96bd-6ce94166c3d0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:46:27.384070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29823d33-4a9d-4c90-ab30-5b888e0c8996 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:46:27.384777 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29823d33-4a9d-4c90-ab30-5b888e0c8996 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:46:27.385295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29823d33-4a9d-4c90-ab30-5b888e0c8996 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:46:27.386381 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29823d33-4a9d-4c90-ab30-5b888e0c8996 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:46:27.388241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-29823d33-4a9d-4c90-ab30-5b888e0c8996 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:46:27.401194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29823d33-4a9d-4c90-ab30-5b888e0c8996 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:46:27.402799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29823d33-4a9d-4c90-ab30-5b888e0c8996 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:46:27.414044 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1382/2748] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:46:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:46:27.430459 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fc073d480884782b4084952345382b9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.443551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fc073d480884782b4084952345382b9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.452949 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fc073d480884782b4084952345382b9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.460424 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fc073d480884782b4084952345382b9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.461103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fc073d480884782b4084952345382b9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.467132 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1367/2749] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:46:27 2022] DELETE /identity/v3/users/5fc073d480884782b4084952345382b9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:46:27.643986 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29823d33-4a9d-4c90-ab30-5b888e0c8996 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:46:27.655524 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29823d33-4a9d-4c90-ab30-5b888e0c8996 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:46:27.715863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:46:27.716954 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:46:27.717299 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:46:27.717671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] PATH_INFO: `/v3/projects/8eb1e5f40033474c947399c22bb0a2d2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:46:27.719096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb1e5f40033474c947399c22bb0a2d2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:46:27.721283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:46:27.772248 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb1e5f40033474c947399c22bb0a2d2', 'request_id': 'req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.780321 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb1e5f40033474c947399c22bb0a2d2', 'request_id': 'req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.785844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb1e5f40033474c947399c22bb0a2d2', 'request_id': 'req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.796845 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] The token cache is being invalidate because project 8eb1e5f40033474c947399c22bb0a2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:46:27.797127 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.797756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.799311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1383/2750] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:46:27 2022] DELETE /identity/v3/projects/8eb1e5f40033474c947399c22bb0a2d2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:46:27.803525 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:46:27.804980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:46:27.818820 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e484b44-1e25-46b9-a2cb-0e6db0d4efd9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:46:27.838865 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:46:27.880722 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:46:27.881655 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:46:27.881942 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:46:27.882209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] PATH_INFO: `/v3/users/c77c4c5e840444929f648cfb735e995f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:46:27.883655 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77c4c5e840444929f648cfb735e995f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:46:27.885583 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:46:27.937920 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77c4c5e840444929f648cfb735e995f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.951964 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77c4c5e840444929f648cfb735e995f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.961807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77c4c5e840444929f648cfb735e995f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.969318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77c4c5e840444929f648cfb735e995f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.969973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77c4c5e840444929f648cfb735e995f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:27.976660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1368/2751] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:46:27 2022] DELETE /identity/v3/users/c77c4c5e840444929f648cfb735e995f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:46:28.160373 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:46:28.161919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:46:28.179389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf346ac1-d0b6-4052-990a-bdec7a2a2a0c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:46:28.197445 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:46:28.239479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:46:28.240355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:46:28.240646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:46:28.240940 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] PATH_INFO: `/v3/projects/de5e5149971141d8ab26dbea50f0cf4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:46:28.242162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5e5149971141d8ab26dbea50f0cf4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:46:28.244074 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:46:28.301051 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5e5149971141d8ab26dbea50f0cf4e', 'request_id': 'req-e71f6715-aa69-483e-9bf4-8676cf54e997'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:28.311887 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5e5149971141d8ab26dbea50f0cf4e', 'request_id': 'req-e71f6715-aa69-483e-9bf4-8676cf54e997'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:28.317574 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5e5149971141d8ab26dbea50f0cf4e', 'request_id': 'req-e71f6715-aa69-483e-9bf4-8676cf54e997'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:28.329437 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] The token cache is being invalidate because project de5e5149971141d8ab26dbea50f0cf4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:46:28.329724 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:28.330499 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:46:28.332179 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1384/2752] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:46:28 2022] DELETE /identity/v3/projects/de5e5149971141d8ab26dbea50f0cf4e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:46:37.973052 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:46:38.326306 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:46:39.308501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:46:39.310302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:46:39.372677 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cc87e-ea6d-4f49-a891-ea233aad7350 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:46:39.427222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4b69e-7093-4890-8d28-61222b24003e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:46:39.427874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4b69e-7093-4890-8d28-61222b24003e service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:46:39.428151 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4b69e-7093-4890-8d28-61222b24003e service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:46:39.428423 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4b69e-7093-4890-8d28-61222b24003e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:46:39.430428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dde4b69e-7093-4890-8d28-61222b24003e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:46:39.440449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4b69e-7093-4890-8d28-61222b24003e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:46:39.441959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4b69e-7093-4890-8d28-61222b24003e service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:46:39.450986 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1369/2753] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:46:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:46:49.447299 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=46 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:05.001720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:05.003300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:05.069450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f6715-aa69-483e-9bf4-8676cf54e997 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:05.074428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:05.075307 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.075589 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.075858 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] PATH_INFO: `/v3/users/c38435f90f9a4cceba68faec58e9a5e4` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.077162 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c38435f90f9a4cceba68faec58e9a5e4)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:05.079100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:05.136664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c38435f90f9a4cceba68faec58e9a5e4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.148830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c38435f90f9a4cceba68faec58e9a5e4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.158645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c38435f90f9a4cceba68faec58e9a5e4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.159269 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c38435f90f9a4cceba68faec58e9a5e4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.166490 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea41ae41-7535-4705-b79f-fbf1e2a0a515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c38435f90f9a4cceba68faec58e9a5e4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.172263 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1385/2754] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:05 2022] DELETE /identity/v3/users/c38435f90f9a4cceba68faec58e9a5e4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:05.361905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4b69e-7093-4890-8d28-61222b24003e service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:05.374767 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4b69e-7093-4890-8d28-61222b24003e service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:05.440392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:05.441323 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.441604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.441885 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] PATH_INFO: `/v3/projects/e946a4548f114b088ac5073b28e31ba0` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.443114 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e946a4548f114b088ac5073b28e31ba0)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:05.445066 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:05.493762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e946a4548f114b088ac5073b28e31ba0', 'request_id': 'req-8bc42124-c866-4d90-90e5-a3af5196734c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.499801 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e946a4548f114b088ac5073b28e31ba0', 'request_id': 'req-8bc42124-c866-4d90-90e5-a3af5196734c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.508407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e946a4548f114b088ac5073b28e31ba0', 'request_id': 'req-8bc42124-c866-4d90-90e5-a3af5196734c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.519251 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] The token cache is being invalidate because project e946a4548f114b088ac5073b28e31ba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:05.519542 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.520176 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8bc42124-c866-4d90-90e5-a3af5196734c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:05.521789 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1370/2755] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:05 2022] DELETE /identity/v3/projects/e946a4548f114b088ac5073b28e31ba0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:05.527028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253275a3-6201-4074-b364-68c563cbefa1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.527253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253275a3-6201-4074-b364-68c563cbefa1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.527529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253275a3-6201-4074-b364-68c563cbefa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.547245 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-253275a3-6201-4074-b364-68c563cbefa1 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:05.548459 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-253275a3-6201-4074-b364-68c563cbefa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:05.613423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1386/2756] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:05.624124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f885b98-d811-4d3b-b2bf-dd22cddac459 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.624414 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f885b98-d811-4d3b-b2bf-dd22cddac459 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.624684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f885b98-d811-4d3b-b2bf-dd22cddac459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.644914 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4f885b98-d811-4d3b-b2bf-dd22cddac459 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:05.646142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4f885b98-d811-4d3b-b2bf-dd22cddac459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:05.655563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1371/2757] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:05.660473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253275a3-6201-4074-b364-68c563cbefa1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:05.672723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253275a3-6201-4074-b364-68c563cbefa1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:05.681537 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb091de-0bb4-466b-8852-ae877f097bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:05.682516 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb091de-0bb4-466b-8852-ae877f097bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.682812 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb091de-0bb4-466b-8852-ae877f097bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.683116 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb091de-0bb4-466b-8852-ae877f097bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.684101 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb091de-0bb4-466b-8852-ae877f097bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:47:05.684463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb091de-0bb4-466b-8852-ae877f097bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:05.686814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb091de-0bb4-466b-8852-ae877f097bf4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:05.700615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1387/2758] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:47:05.705412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f885b98-d811-4d3b-b2bf-dd22cddac459 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:05.715588 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f885b98-d811-4d3b-b2bf-dd22cddac459 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:05.721645 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90fee0-2c00-4118-860e-df2244d84fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:05.722638 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90fee0-2c00-4118-860e-df2244d84fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.722926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90fee0-2c00-4118-860e-df2244d84fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.723197 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90fee0-2c00-4118-860e-df2244d84fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.723889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90fee0-2c00-4118-860e-df2244d84fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:05.726179 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90fee0-2c00-4118-860e-df2244d84fb6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:05.755858 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1372/2759] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:05 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:05.760241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb091de-0bb4-466b-8852-ae877f097bf4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:05.770919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb091de-0bb4-466b-8852-ae877f097bf4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:05.774938 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:05.834038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93959289-b3f2-4a79-b55a-9809a6fbb49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:05.834620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93959289-b3f2-4a79-b55a-9809a6fbb49a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.834898 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93959289-b3f2-4a79-b55a-9809a6fbb49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.835175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93959289-b3f2-4a79-b55a-9809a6fbb49a admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.835856 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93959289-b3f2-4a79-b55a-9809a6fbb49a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:05.837822 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93959289-b3f2-4a79-b55a-9809a6fbb49a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:05.860253 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1388/2760] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:05 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:05.864543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90fee0-2c00-4118-860e-df2244d84fb6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:05.880544 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90fee0-2c00-4118-860e-df2244d84fb6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:05.886740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea83274-cf66-44f3-bcbe-5d7f54b01e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:05.887405 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea83274-cf66-44f3-bcbe-5d7f54b01e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.887692 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea83274-cf66-44f3-bcbe-5d7f54b01e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.887969 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea83274-cf66-44f3-bcbe-5d7f54b01e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.888701 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea83274-cf66-44f3-bcbe-5d7f54b01e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:05.890337 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea83274-cf66-44f3-bcbe-5d7f54b01e16 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:05.901410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1373/2761] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:05.905941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93959289-b3f2-4a79-b55a-9809a6fbb49a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:05.916302 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93959289-b3f2-4a79-b55a-9809a6fbb49a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:05.921355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285681b4-3e8d-430e-af0d-da667460aec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:05.922017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285681b4-3e8d-430e-af0d-da667460aec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.922305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285681b4-3e8d-430e-af0d-da667460aec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.922556 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285681b4-3e8d-430e-af0d-da667460aec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.923293 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285681b4-3e8d-430e-af0d-da667460aec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:05.924967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285681b4-3e8d-430e-af0d-da667460aec9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:05.935420 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1389/2762] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:05.939581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea83274-cf66-44f3-bcbe-5d7f54b01e16 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:05.949913 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea83274-cf66-44f3-bcbe-5d7f54b01e16 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:05.954510 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228845d-7532-40a9-b3fa-fab2989fe9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:05.955390 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228845d-7532-40a9-b3fa-fab2989fe9ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:05.955673 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228845d-7532-40a9-b3fa-fab2989fe9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:05.955938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228845d-7532-40a9-b3fa-fab2989fe9ee admin admin] PATH_INFO: `/v3/projects/9e41a51a179b4371a2815bb2d02ea1d7/users/62527016dd54474dae5d1164f47b6d24/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:05.970368 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228845d-7532-40a9-b3fa-fab2989fe9ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e41a51a179b4371a2815bb2d02ea1d7, user_id=62527016dd54474dae5d1164f47b6d24, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:05.973480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228845d-7532-40a9-b3fa-fab2989fe9ee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:05.988900 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1374/2763] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:05 2022] PUT /identity/v3/projects/9e41a51a179b4371a2815bb2d02ea1d7/users/62527016dd54474dae5d1164f47b6d24/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:06.000182 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffbe9c-ee3c-446f-9a97-2c2e3b00c821 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:06.000480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffbe9c-ee3c-446f-9a97-2c2e3b00c821 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:06.000790 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffbe9c-ee3c-446f-9a97-2c2e3b00c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:06.030661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-05ffbe9c-ee3c-446f-9a97-2c2e3b00c821 None None] MFA Rules not processed for user `62527016dd54474dae5d1164f47b6d24`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:06.032205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-05ffbe9c-ee3c-446f-9a97-2c2e3b00c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:06.107117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1390/2764] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:06.118551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228845d-7532-40a9-b3fa-fab2989fe9ee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:06.120404 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6228845d-7532-40a9-b3fa-fab2989fe9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:06.152283 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:06.207158 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228845d-7532-40a9-b3fa-fab2989fe9ee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:06.213591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadac369-d0d0-4316-95e2-e8b53884acd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:06.214243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadac369-d0d0-4316-95e2-e8b53884acd3 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:06.214560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadac369-d0d0-4316-95e2-e8b53884acd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:06.214843 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadac369-d0d0-4316-95e2-e8b53884acd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:06.223406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadac369-d0d0-4316-95e2-e8b53884acd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:06.225038 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadac369-d0d0-4316-95e2-e8b53884acd3 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:06.236916 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1375/2765] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:06.453959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffbe9c-ee3c-446f-9a97-2c2e3b00c821 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:06.521911 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffbe9c-ee3c-446f-9a97-2c2e3b00c821 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:06.526702 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:06.527595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:06.527881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:06.528159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] PATH_INFO: `/v3/users/62527016dd54474dae5d1164f47b6d24` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:06.529413 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62527016dd54474dae5d1164f47b6d24)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:06.531356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:06.548474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:06.585009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62527016dd54474dae5d1164f47b6d24'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:06.598726 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62527016dd54474dae5d1164f47b6d24'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:06.608286 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62527016dd54474dae5d1164f47b6d24'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:06.609038 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62527016dd54474dae5d1164f47b6d24'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:06.615948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62527016dd54474dae5d1164f47b6d24'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:06.622016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1391/2766] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:06 2022] DELETE /identity/v3/users/62527016dd54474dae5d1164f47b6d24 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:06.626734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ed740-6b3a-47b5-8a61-61517b7d678c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:06.627217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ed740-6b3a-47b5-8a61-61517b7d678c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:06.627317 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ed740-6b3a-47b5-8a61-61517b7d678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:06.645629 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a95ed740-6b3a-47b5-8a61-61517b7d678c None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:06.646854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a95ed740-6b3a-47b5-8a61-61517b7d678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:06.714782 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1376/2767] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:06.725057 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:06.726620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:06.784819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660bf8d-4af9-49f4-b592-c2bbfc590808 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:06.789067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd567e-b75d-46dd-88a4-7aef9e433958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:06.789714 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd567e-b75d-46dd-88a4-7aef9e433958 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:06.789993 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd567e-b75d-46dd-88a4-7aef9e433958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:06.790274 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd567e-b75d-46dd-88a4-7aef9e433958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:06.799662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd567e-b75d-46dd-88a4-7aef9e433958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:06.801156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd567e-b75d-46dd-88a4-7aef9e433958 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:06.810650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1392/2768] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:07.088239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ed740-6b3a-47b5-8a61-61517b7d678c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.089862 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a95ed740-6b3a-47b5-8a61-61517b7d678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:07.104420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ed740-6b3a-47b5-8a61-61517b7d678c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.109395 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.110372 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.110754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.111243 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] PATH_INFO: `/v3/projects/9e41a51a179b4371a2815bb2d02ea1d7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.112403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e41a51a179b4371a2815bb2d02ea1d7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.114330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.162240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e41a51a179b4371a2815bb2d02ea1d7', 'request_id': 'req-54a7ea09-d877-4019-aef7-a31445909666'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:07.168278 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e41a51a179b4371a2815bb2d02ea1d7', 'request_id': 'req-54a7ea09-d877-4019-aef7-a31445909666'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:07.177416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e41a51a179b4371a2815bb2d02ea1d7', 'request_id': 'req-54a7ea09-d877-4019-aef7-a31445909666'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:07.187840 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] The token cache is being invalidate because project 9e41a51a179b4371a2815bb2d02ea1d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:07.188124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:07.188749 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-54a7ea09-d877-4019-aef7-a31445909666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:07.190301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1377/2769] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:07 2022] DELETE /identity/v3/projects/9e41a51a179b4371a2815bb2d02ea1d7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:07.195311 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8acfb-3feb-4ef0-bb66-0c4a75f76657 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.195813 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8acfb-3feb-4ef0-bb66-0c4a75f76657 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.195910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8acfb-3feb-4ef0-bb66-0c4a75f76657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.214152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b8c8acfb-3feb-4ef0-bb66-0c4a75f76657 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:07.215356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b8c8acfb-3feb-4ef0-bb66-0c4a75f76657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:07.278329 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1393/2770] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:07.286259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa86c0-da16-42d8-b060-38de2b06fdae None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.286547 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa86c0-da16-42d8-b060-38de2b06fdae None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.286823 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa86c0-da16-42d8-b060-38de2b06fdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.305232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b2fa86c0-da16-42d8-b060-38de2b06fdae None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:07.306422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b2fa86c0-da16-42d8-b060-38de2b06fdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:07.317200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1378/2771] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:07.319111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8acfb-3feb-4ef0-bb66-0c4a75f76657 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.329161 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8acfb-3feb-4ef0-bb66-0c4a75f76657 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.333954 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0bf0e-236a-4a8e-b6ee-9b274f7121f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.334605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0bf0e-236a-4a8e-b6ee-9b274f7121f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.334888 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0bf0e-236a-4a8e-b6ee-9b274f7121f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.335166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0bf0e-236a-4a8e-b6ee-9b274f7121f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.335892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0bf0e-236a-4a8e-b6ee-9b274f7121f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:47:07.336205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0bf0e-236a-4a8e-b6ee-9b274f7121f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.337906 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0bf0e-236a-4a8e-b6ee-9b274f7121f7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.350603 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1394/2772] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:47:07.354905 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa86c0-da16-42d8-b060-38de2b06fdae None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.364152 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa86c0-da16-42d8-b060-38de2b06fdae None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.369403 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6a9fa-a588-429c-b420-98eb2b639f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.370048 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6a9fa-a588-429c-b420-98eb2b639f60 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.370325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6a9fa-a588-429c-b420-98eb2b639f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.370593 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6a9fa-a588-429c-b420-98eb2b639f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.371327 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6a9fa-a588-429c-b420-98eb2b639f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.373240 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6a9fa-a588-429c-b420-98eb2b639f60 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.406077 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1379/2773] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:07 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:07.408229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0bf0e-236a-4a8e-b6ee-9b274f7121f7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.418440 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0bf0e-236a-4a8e-b6ee-9b274f7121f7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.438552 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:07.478027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e019c2c-a6e9-4399-89d8-dd05040fbb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.478712 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e019c2c-a6e9-4399-89d8-dd05040fbb32 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.478990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e019c2c-a6e9-4399-89d8-dd05040fbb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.479267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e019c2c-a6e9-4399-89d8-dd05040fbb32 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.479957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e019c2c-a6e9-4399-89d8-dd05040fbb32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.481910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e019c2c-a6e9-4399-89d8-dd05040fbb32 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.507303 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1395/2774] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:07 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:07.511321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6a9fa-a588-429c-b420-98eb2b639f60 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.522276 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6a9fa-a588-429c-b420-98eb2b639f60 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.527930 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37cd0d-d6d2-4523-b746-9368ecc7f8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.528624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37cd0d-d6d2-4523-b746-9368ecc7f8ad admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.528943 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37cd0d-d6d2-4523-b746-9368ecc7f8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.529213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37cd0d-d6d2-4523-b746-9368ecc7f8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.529939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37cd0d-d6d2-4523-b746-9368ecc7f8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.531523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37cd0d-d6d2-4523-b746-9368ecc7f8ad admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.532816 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:07.544354 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1380/2775] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:07.548513 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e019c2c-a6e9-4399-89d8-dd05040fbb32 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.559244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e019c2c-a6e9-4399-89d8-dd05040fbb32 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.564082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcf2be-d67c-490f-9170-e7b8fc27cd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.564766 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcf2be-d67c-490f-9170-e7b8fc27cd2a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.565068 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcf2be-d67c-490f-9170-e7b8fc27cd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.565355 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcf2be-d67c-490f-9170-e7b8fc27cd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.566110 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcf2be-d67c-490f-9170-e7b8fc27cd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.567717 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcf2be-d67c-490f-9170-e7b8fc27cd2a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.578332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1396/2776] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:07.582546 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37cd0d-d6d2-4523-b746-9368ecc7f8ad admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.595523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37cd0d-d6d2-4523-b746-9368ecc7f8ad admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.601618 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7970d-bf13-45e5-a78c-f125ec0f8fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.602492 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7970d-bf13-45e5-a78c-f125ec0f8fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.602770 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7970d-bf13-45e5-a78c-f125ec0f8fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.603045 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7970d-bf13-45e5-a78c-f125ec0f8fd7 admin admin] PATH_INFO: `/v3/projects/e9862baa0e1e4ed18d2888fed897d0c7/users/954b1b51f17f463b91021fb551b99a2f/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.618389 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7970d-bf13-45e5-a78c-f125ec0f8fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9862baa0e1e4ed18d2888fed897d0c7, user_id=954b1b51f17f463b91021fb551b99a2f, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.621171 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7970d-bf13-45e5-a78c-f125ec0f8fd7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.640321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1381/2777] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:07 2022] PUT /identity/v3/projects/e9862baa0e1e4ed18d2888fed897d0c7/users/954b1b51f17f463b91021fb551b99a2f/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:07.651056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6328e00-6d31-4662-9e9f-f0fad5dab48e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.651561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6328e00-6d31-4662-9e9f-f0fad5dab48e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.651925 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6328e00-6d31-4662-9e9f-f0fad5dab48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.680442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c6328e00-6d31-4662-9e9f-f0fad5dab48e None None] MFA Rules not processed for user `954b1b51f17f463b91021fb551b99a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:07.681841 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c6328e00-6d31-4662-9e9f-f0fad5dab48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:07.747792 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1397/2778] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:07.752526 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7970d-bf13-45e5-a78c-f125ec0f8fd7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.817065 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7970d-bf13-45e5-a78c-f125ec0f8fd7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.821939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce06d-d9bc-43c7-abc7-fb9a1d553bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.822815 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce06d-d9bc-43c7-abc7-fb9a1d553bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.823112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce06d-d9bc-43c7-abc7-fb9a1d553bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.823382 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce06d-d9bc-43c7-abc7-fb9a1d553bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.824115 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fce06d-d9bc-43c7-abc7-fb9a1d553bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.826039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fce06d-d9bc-43c7-abc7-fb9a1d553bd2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.854597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1382/2779] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:07 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:07.858838 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6328e00-6d31-4662-9e9f-f0fad5dab48e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.870016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6328e00-6d31-4662-9e9f-f0fad5dab48e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.924963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687ad27-d435-47df-b9fa-c3394bca89ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.925623 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687ad27-d435-47df-b9fa-c3394bca89ed admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.925905 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687ad27-d435-47df-b9fa-c3394bca89ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.926181 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687ad27-d435-47df-b9fa-c3394bca89ed admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.926872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687ad27-d435-47df-b9fa-c3394bca89ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.928846 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687ad27-d435-47df-b9fa-c3394bca89ed admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.952193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1398/2780] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:07 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:07.956271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce06d-d9bc-43c7-abc7-fb9a1d553bd2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:07.970490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce06d-d9bc-43c7-abc7-fb9a1d553bd2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:07.975478 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c613ba4-1bed-43b5-b645-c95427849dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:07.976133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c613ba4-1bed-43b5-b645-c95427849dba admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:07.976412 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c613ba4-1bed-43b5-b645-c95427849dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:07.976683 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c613ba4-1bed-43b5-b645-c95427849dba admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:07.977446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c613ba4-1bed-43b5-b645-c95427849dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:07.979124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c613ba4-1bed-43b5-b645-c95427849dba admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:07.989772 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1383/2781] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:07.993937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687ad27-d435-47df-b9fa-c3394bca89ed admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:08.003990 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687ad27-d435-47df-b9fa-c3394bca89ed admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:08.008983 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d91e8-2440-4cc0-9f7c-58df979c61b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:08.010187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d91e8-2440-4cc0-9f7c-58df979c61b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:08.010187 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d91e8-2440-4cc0-9f7c-58df979c61b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:08.010427 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d91e8-2440-4cc0-9f7c-58df979c61b3 admin admin] PATH_INFO: `/v3/projects/cab64e16c71944b395223d5b905bdb90/users/6cac84799fce4e11b30683eed1fc0d0d/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:08.025529 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d91e8-2440-4cc0-9f7c-58df979c61b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab64e16c71944b395223d5b905bdb90, user_id=6cac84799fce4e11b30683eed1fc0d0d, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:08.028060 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d91e8-2440-4cc0-9f7c-58df979c61b3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:08.041165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:08.047491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1399/2782] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:07 2022] PUT /identity/v3/projects/cab64e16c71944b395223d5b905bdb90/users/6cac84799fce4e11b30683eed1fc0d0d/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:47:08.051331 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c613ba4-1bed-43b5-b645-c95427849dba admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:08.061686 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c613ba4-1bed-43b5-b645-c95427849dba admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:08.130733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971127d-24fe-4fe0-9228-c0248610c66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:08.131624 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971127d-24fe-4fe0-9228-c0248610c66e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:08.131938 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971127d-24fe-4fe0-9228-c0248610c66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:08.132321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971127d-24fe-4fe0-9228-c0248610c66e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:08.133146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971127d-24fe-4fe0-9228-c0248610c66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:08.135244 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971127d-24fe-4fe0-9228-c0248610c66e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:08.146005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1384/2783] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:08.150644 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d91e8-2440-4cc0-9f7c-58df979c61b3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:08.161528 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d91e8-2440-4cc0-9f7c-58df979c61b3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:08.166697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289976e-1c43-4a42-8ed1-b1dc9045a97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:08.167591 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289976e-1c43-4a42-8ed1-b1dc9045a97e admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:08.167881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289976e-1c43-4a42-8ed1-b1dc9045a97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:08.168159 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289976e-1c43-4a42-8ed1-b1dc9045a97e admin admin] PATH_INFO: `/v3/projects/cab64e16c71944b395223d5b905bdb90/users/6cac84799fce4e11b30683eed1fc0d0d/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:08.170198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289976e-1c43-4a42-8ed1-b1dc9045a97e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab64e16c71944b395223d5b905bdb90, user_id=6cac84799fce4e11b30683eed1fc0d0d, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:08.172881 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289976e-1c43-4a42-8ed1-b1dc9045a97e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:08.183561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1400/2784] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:08 2022] PUT /identity/v3/projects/cab64e16c71944b395223d5b905bdb90/users/6cac84799fce4e11b30683eed1fc0d0d/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:08.195959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bd67c-2c29-4d6b-a50e-ee084ab490a4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:08.196361 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bd67c-2c29-4d6b-a50e-ee084ab490a4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:08.196663 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bd67c-2c29-4d6b-a50e-ee084ab490a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:08.229101 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-676bd67c-2c29-4d6b-a50e-ee084ab490a4 None None] MFA Rules not processed for user `6cac84799fce4e11b30683eed1fc0d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:08.230262 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-676bd67c-2c29-4d6b-a50e-ee084ab490a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:08.261433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:08.314321 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1385/2785] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:08.327665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289976e-1c43-4a42-8ed1-b1dc9045a97e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:08.329824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c289976e-1c43-4a42-8ed1-b1dc9045a97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:08.392525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289976e-1c43-4a42-8ed1-b1dc9045a97e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:08.396180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb14dbf-bfec-4718-85b3-f336dd5da00f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:08.396854 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb14dbf-bfec-4718-85b3-f336dd5da00f service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:08.397160 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb14dbf-bfec-4718-85b3-f336dd5da00f service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:08.397435 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb14dbf-bfec-4718-85b3-f336dd5da00f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:08.411366 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb14dbf-bfec-4718-85b3-f336dd5da00f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:08.413679 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb14dbf-bfec-4718-85b3-f336dd5da00f service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:08.429208 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1401/2786] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:08.766428 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bd67c-2c29-4d6b-a50e-ee084ab490a4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:08.775886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bd67c-2c29-4d6b-a50e-ee084ab490a4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:08.781719 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93d652-0da7-4307-872b-7efc6accdc61 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cac84799fce4e11b30683eed1fc0d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab64e16c71944b395223d5b905bdb90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:08.782366 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93d652-0da7-4307-872b-7efc6accdc61 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:08.782687 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93d652-0da7-4307-872b-7efc6accdc61 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:08.782967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93d652-0da7-4307-872b-7efc6accdc61 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:08.783705 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93d652-0da7-4307-872b-7efc6accdc61 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:47:08.784016 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93d652-0da7-4307-872b-7efc6accdc61 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:08.785682 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93d652-0da7-4307-872b-7efc6accdc61 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:08.797504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1386/2787] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:47:08.801620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb14dbf-bfec-4718-85b3-f336dd5da00f service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:08.811221 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb14dbf-bfec-4718-85b3-f336dd5da00f service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:08.817019 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba59c9-0010-4888-afd0-e544e006f8bb tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cac84799fce4e11b30683eed1fc0d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab64e16c71944b395223d5b905bdb90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:08.817671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba59c9-0010-4888-afd0-e544e006f8bb tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:08.817957 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba59c9-0010-4888-afd0-e544e006f8bb tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:08.818238 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba59c9-0010-4888-afd0-e544e006f8bb tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:08.818933 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba59c9-0010-4888-afd0-e544e006f8bb tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:08.820861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba59c9-0010-4888-afd0-e544e006f8bb tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:08.850907 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1402/2788] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:08 2022] POST /identity/v3/projects => generated 354 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:08.855153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93d652-0da7-4307-872b-7efc6accdc61 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:08.865475 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93d652-0da7-4307-872b-7efc6accdc61 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:08.922490 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cac84799fce4e11b30683eed1fc0d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab64e16c71944b395223d5b905bdb90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:08.923157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:08.923446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:08.923728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:08.924455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:47:08.924753 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:08.926418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:08.938824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1387/2789] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:47:08.974165 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba59c9-0010-4888-afd0-e544e006f8bb tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:08.984496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba59c9-0010-4888-afd0-e544e006f8bb tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:08.990016 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cac84799fce4e11b30683eed1fc0d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab64e16c71944b395223d5b905bdb90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:08.990888 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:08.991154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:08.991451 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] PATH_INFO: `/v3/projects/b2bdac19062b41d8afc4b2675f1d8f0e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:08.992756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b2bdac19062b41d8afc4b2675f1d8f0e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:08.994959 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:09.045640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2bdac19062b41d8afc4b2675f1d8f0e', 'request_id': 'req-df49021a-b9b7-4cbc-86e9-5127140bd98b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.055059 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2bdac19062b41d8afc4b2675f1d8f0e', 'request_id': 'req-df49021a-b9b7-4cbc-86e9-5127140bd98b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.061147 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2bdac19062b41d8afc4b2675f1d8f0e', 'request_id': 'req-df49021a-b9b7-4cbc-86e9-5127140bd98b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.072072 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] The token cache is being invalidate because project b2bdac19062b41d8afc4b2675f1d8f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:09.072412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.073218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-df49021a-b9b7-4cbc-86e9-5127140bd98b tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.074802 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1403/2790] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:08 2022] DELETE /identity/v3/projects/b2bdac19062b41d8afc4b2675f1d8f0e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:09.293724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:09.296972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:09.368425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ae9e9-1d67-42ff-9162-a21cb9405aa9 tempest-VolumeQuotaClassesTest-1667221562 tempest-VolumeQuotaClassesTest-1667221562-project-admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:09.422133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:09.429393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:09.430303 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:09.430586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:09.430861 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] PATH_INFO: `/v3/users/954b1b51f17f463b91021fb551b99a2f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:09.432350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=954b1b51f17f463b91021fb551b99a2f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:09.434333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:09.484500 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '954b1b51f17f463b91021fb551b99a2f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.496103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '954b1b51f17f463b91021fb551b99a2f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.505480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '954b1b51f17f463b91021fb551b99a2f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.512996 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '954b1b51f17f463b91021fb551b99a2f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.513723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '954b1b51f17f463b91021fb551b99a2f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:09.519710 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1388/2791] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:09 2022] DELETE /identity/v3/users/954b1b51f17f463b91021fb551b99a2f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:09.524487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612165b-768b-4f20-a267-c867ff239c7b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:09.525003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612165b-768b-4f20-a267-c867ff239c7b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:09.525003 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612165b-768b-4f20-a267-c867ff239c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:09.544671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0612165b-768b-4f20-a267-c867ff239c7b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:09.545948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0612165b-768b-4f20-a267-c867ff239c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:09.590992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:09.608173 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1404/2792] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:09.618157 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:09.619903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:09.688908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0af4fe-4ea8-44c1-bfa1-79c3eb1f5738 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:09.692886 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8bb9e-3588-4a14-b144-8b2307ac7973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:09.693573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8bb9e-3588-4a14-b144-8b2307ac7973 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:09.693897 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8bb9e-3588-4a14-b144-8b2307ac7973 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:09.694084 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8bb9e-3588-4a14-b144-8b2307ac7973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:09.703738 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8bb9e-3588-4a14-b144-8b2307ac7973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:09.705206 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8bb9e-3588-4a14-b144-8b2307ac7973 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:09.714959 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1389/2793] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:10.006934 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612165b-768b-4f20-a267-c867ff239c7b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:10.008533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0612165b-768b-4f20-a267-c867ff239c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:10.021605 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612165b-768b-4f20-a267-c867ff239c7b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:10.027369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:10.028367 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:10.028796 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:10.029275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] PATH_INFO: `/v3/projects/e9862baa0e1e4ed18d2888fed897d0c7` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:10.030773 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9862baa0e1e4ed18d2888fed897d0c7)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:10.033526 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:10.084069 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9862baa0e1e4ed18d2888fed897d0c7', 'request_id': 'req-8582e89d-5a6c-4e04-bd1b-038d01eadc53'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.093180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9862baa0e1e4ed18d2888fed897d0c7', 'request_id': 'req-8582e89d-5a6c-4e04-bd1b-038d01eadc53'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.099391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9862baa0e1e4ed18d2888fed897d0c7', 'request_id': 'req-8582e89d-5a6c-4e04-bd1b-038d01eadc53'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.110442 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] The token cache is being invalidate because project e9862baa0e1e4ed18d2888fed897d0c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:10.110727 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.111415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.113005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1405/2794] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:10 2022] DELETE /identity/v3/projects/e9862baa0e1e4ed18d2888fed897d0c7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:10.117156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8bb9e-3588-4a14-b144-8b2307ac7973 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:10.127577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8bb9e-3588-4a14-b144-8b2307ac7973 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:10.188250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:10.189156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:10.189436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:10.189707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] PATH_INFO: `/v3/users/6cac84799fce4e11b30683eed1fc0d0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:10.190948 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cac84799fce4e11b30683eed1fc0d0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:10.193027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:10.214851 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:10.245985 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cac84799fce4e11b30683eed1fc0d0d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.258333 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cac84799fce4e11b30683eed1fc0d0d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.267325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cac84799fce4e11b30683eed1fc0d0d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.274374 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cac84799fce4e11b30683eed1fc0d0d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.275039 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1b1aa1c0-2bb3-4161-86ee-2c5a95f3aa07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cac84799fce4e11b30683eed1fc0d0d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.281209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1390/2795] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:10 2022] DELETE /identity/v3/users/6cac84799fce4e11b30683eed1fc0d0d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:10.563297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:10.565233 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:10.581932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582e89d-5a6c-4e04-bd1b-038d01eadc53 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:10.650029 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:10.650963 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:10.651280 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:10.651560 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] PATH_INFO: `/v3/projects/cab64e16c71944b395223d5b905bdb90` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:10.652782 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab64e16c71944b395223d5b905bdb90)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:10.654693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:10.703192 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab64e16c71944b395223d5b905bdb90', 'request_id': 'req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.713364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab64e16c71944b395223d5b905bdb90', 'request_id': 'req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.719108 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab64e16c71944b395223d5b905bdb90', 'request_id': 'req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.730686 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] The token cache is being invalidate because project cab64e16c71944b395223d5b905bdb90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:10.730978 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.732128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-606ecc3b-f3e7-49f9-b4c8-f6ef6676ba33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:10.734002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1406/2796] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:10 2022] DELETE /identity/v3/projects/cab64e16c71944b395223d5b905bdb90 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:10.739081 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6340ded-6ed8-43c6-b851-eb4753709a2a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:10.739386 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6340ded-6ed8-43c6-b851-eb4753709a2a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:10.739691 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6340ded-6ed8-43c6-b851-eb4753709a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:10.759939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d6340ded-6ed8-43c6-b851-eb4753709a2a None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:10.761200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d6340ded-6ed8-43c6-b851-eb4753709a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:10.823477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1391/2797] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:10.831561 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74aa15-b356-437a-84d9-715d757c3a0b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:10.831851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74aa15-b356-437a-84d9-715d757c3a0b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:10.832132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74aa15-b356-437a-84d9-715d757c3a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:10.839876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:10.854755 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9c74aa15-b356-437a-84d9-715d757c3a0b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:10.856519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9c74aa15-b356-437a-84d9-715d757c3a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:10.866195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1407/2798] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:10.870811 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6340ded-6ed8-43c6-b851-eb4753709a2a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:10.881127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6340ded-6ed8-43c6-b851-eb4753709a2a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:10.886989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc9d3a-fa70-4237-932d-af9ae3a788b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:10.887648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9d3a-fa70-4237-932d-af9ae3a788b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:10.887929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9d3a-fa70-4237-932d-af9ae3a788b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:10.888199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9d3a-fa70-4237-932d-af9ae3a788b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:10.888973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc9d3a-fa70-4237-932d-af9ae3a788b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:47:10.889279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc9d3a-fa70-4237-932d-af9ae3a788b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:10.890883 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc9d3a-fa70-4237-932d-af9ae3a788b4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:10.903116 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1392/2799] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:47:10.907419 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74aa15-b356-437a-84d9-715d757c3a0b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:10.917260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74aa15-b356-437a-84d9-715d757c3a0b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:10.922195 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1bf2b-793e-4ab6-b1e1-57e1b47d588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:10.922849 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1bf2b-793e-4ab6-b1e1-57e1b47d588d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:10.923130 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1bf2b-793e-4ab6-b1e1-57e1b47d588d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:10.923411 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1bf2b-793e-4ab6-b1e1-57e1b47d588d admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:10.924096 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1bf2b-793e-4ab6-b1e1-57e1b47d588d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:10.926033 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1bf2b-793e-4ab6-b1e1-57e1b47d588d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:10.954982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1408/2800] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:10 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:10.960990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc9d3a-fa70-4237-932d-af9ae3a788b4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:10.973740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc9d3a-fa70-4237-932d-af9ae3a788b4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.030998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b3a0a-b3f7-4764-97c7-01485f4854ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.031670 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b3a0a-b3f7-4764-97c7-01485f4854ef admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.031952 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b3a0a-b3f7-4764-97c7-01485f4854ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.032227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b3a0a-b3f7-4764-97c7-01485f4854ef admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.032945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b3a0a-b3f7-4764-97c7-01485f4854ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.034854 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b3a0a-b3f7-4764-97c7-01485f4854ef admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.063810 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1393/2801] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:10 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:47:11.069653 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1bf2b-793e-4ab6-b1e1-57e1b47d588d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.082005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1bf2b-793e-4ab6-b1e1-57e1b47d588d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.086697 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746695f-e4e0-4a64-92e4-72317d72880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.087356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746695f-e4e0-4a64-92e4-72317d72880d admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.087643 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746695f-e4e0-4a64-92e4-72317d72880d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.087919 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746695f-e4e0-4a64-92e4-72317d72880d admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.088642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746695f-e4e0-4a64-92e4-72317d72880d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.090346 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746695f-e4e0-4a64-92e4-72317d72880d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.101539 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1409/2802] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:11.106198 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b3a0a-b3f7-4764-97c7-01485f4854ef admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.116308 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b3a0a-b3f7-4764-97c7-01485f4854ef admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.121341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e339e25-d622-4b44-a0a6-db7c4a48de5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.122122 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e339e25-d622-4b44-a0a6-db7c4a48de5d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.122408 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e339e25-d622-4b44-a0a6-db7c4a48de5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.122684 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e339e25-d622-4b44-a0a6-db7c4a48de5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.123430 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e339e25-d622-4b44-a0a6-db7c4a48de5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.125047 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e339e25-d622-4b44-a0a6-db7c4a48de5d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.135413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1394/2803] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:11.139721 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746695f-e4e0-4a64-92e4-72317d72880d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.150444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746695f-e4e0-4a64-92e4-72317d72880d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.157517 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740a17b-764f-49e7-b89d-c1334a559a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.158471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740a17b-764f-49e7-b89d-c1334a559a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.163502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740a17b-764f-49e7-b89d-c1334a559a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.163502 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740a17b-764f-49e7-b89d-c1334a559a79 admin admin] PATH_INFO: `/v3/projects/27432a933d2e45a3940fd491c50f66c3/users/6809f80b3362418696e88d3eafe95748/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.177034 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740a17b-764f-49e7-b89d-c1334a559a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27432a933d2e45a3940fd491c50f66c3, user_id=6809f80b3362418696e88d3eafe95748, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.179711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740a17b-764f-49e7-b89d-c1334a559a79 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.196487 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1410/2804] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:11 2022] PUT /identity/v3/projects/27432a933d2e45a3940fd491c50f66c3/users/6809f80b3362418696e88d3eafe95748/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:11.207034 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad3055-69b9-44b6-8f04-cff7a40450e4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.207328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad3055-69b9-44b6-8f04-cff7a40450e4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.207610 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad3055-69b9-44b6-8f04-cff7a40450e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.214548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:11.239295 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7dad3055-69b9-44b6-8f04-cff7a40450e4 None None] MFA Rules not processed for user `6809f80b3362418696e88d3eafe95748`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:11.240504 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7dad3055-69b9-44b6-8f04-cff7a40450e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:11.307632 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1395/2805] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:47:11.312219 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740a17b-764f-49e7-b89d-c1334a559a79 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.377819 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740a17b-764f-49e7-b89d-c1334a559a79 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.383081 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224fda5-a3ba-4d8b-b17a-408e841808d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.383742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224fda5-a3ba-4d8b-b17a-408e841808d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.384022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224fda5-a3ba-4d8b-b17a-408e841808d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.384297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224fda5-a3ba-4d8b-b17a-408e841808d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.385098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224fda5-a3ba-4d8b-b17a-408e841808d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.386908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224fda5-a3ba-4d8b-b17a-408e841808d5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.417786 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1411/2806] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:11 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:11.421972 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad3055-69b9-44b6-8f04-cff7a40450e4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.434376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad3055-69b9-44b6-8f04-cff7a40450e4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.495573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f80fe-3ed5-43b8-9e1f-9f6a288c0bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.496226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f80fe-3ed5-43b8-9e1f-9f6a288c0bdb admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.496515 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f80fe-3ed5-43b8-9e1f-9f6a288c0bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.496824 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f80fe-3ed5-43b8-9e1f-9f6a288c0bdb admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.497511 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f80fe-3ed5-43b8-9e1f-9f6a288c0bdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.499433 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f80fe-3ed5-43b8-9e1f-9f6a288c0bdb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.523185 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1396/2807] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:11 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:11.527213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224fda5-a3ba-4d8b-b17a-408e841808d5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.538241 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224fda5-a3ba-4d8b-b17a-408e841808d5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.542994 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bc846-006c-41f7-959e-5e657b966522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.543651 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bc846-006c-41f7-959e-5e657b966522 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.543937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bc846-006c-41f7-959e-5e657b966522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.544210 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bc846-006c-41f7-959e-5e657b966522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.544969 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bc846-006c-41f7-959e-5e657b966522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.546595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bc846-006c-41f7-959e-5e657b966522 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.557358 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1412/2808] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:11.562259 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f80fe-3ed5-43b8-9e1f-9f6a288c0bdb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.576226 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f80fe-3ed5-43b8-9e1f-9f6a288c0bdb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.582107 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c5c0d-1afe-402a-8ea2-df86a287f271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.582989 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c5c0d-1afe-402a-8ea2-df86a287f271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.583290 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c5c0d-1afe-402a-8ea2-df86a287f271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.583559 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c5c0d-1afe-402a-8ea2-df86a287f271 admin admin] PATH_INFO: `/v3/projects/f3207a01779a4397b1beaebbc38530ba/users/d146463b8d944bb0ab517d0d613eef64/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.599477 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c5c0d-1afe-402a-8ea2-df86a287f271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3207a01779a4397b1beaebbc38530ba, user_id=d146463b8d944bb0ab517d0d613eef64, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.602161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c5c0d-1afe-402a-8ea2-df86a287f271 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.617773 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1397/2809] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:11 2022] PUT /identity/v3/projects/f3207a01779a4397b1beaebbc38530ba/users/d146463b8d944bb0ab517d0d613eef64/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:11.621880 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bc846-006c-41f7-959e-5e657b966522 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.631851 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bc846-006c-41f7-959e-5e657b966522 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.689023 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86633d7b-6aad-4bed-b806-96650cf82af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.689601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86633d7b-6aad-4bed-b806-96650cf82af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.689864 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86633d7b-6aad-4bed-b806-96650cf82af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.690134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86633d7b-6aad-4bed-b806-96650cf82af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.690853 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86633d7b-6aad-4bed-b806-96650cf82af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.692462 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86633d7b-6aad-4bed-b806-96650cf82af2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.702371 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1413/2810] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:47:11.706641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c5c0d-1afe-402a-8ea2-df86a287f271 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.717257 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c5c0d-1afe-402a-8ea2-df86a287f271 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.722533 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab8057-ccc4-4669-ba83-a4fae45f6980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.723416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab8057-ccc4-4669-ba83-a4fae45f6980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.723685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab8057-ccc4-4669-ba83-a4fae45f6980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.723958 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab8057-ccc4-4669-ba83-a4fae45f6980 admin admin] PATH_INFO: `/v3/projects/f3207a01779a4397b1beaebbc38530ba/users/d146463b8d944bb0ab517d0d613eef64/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.725939 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab8057-ccc4-4669-ba83-a4fae45f6980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3207a01779a4397b1beaebbc38530ba, user_id=d146463b8d944bb0ab517d0d613eef64, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.728660 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab8057-ccc4-4669-ba83-a4fae45f6980 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.740269 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1398/2811] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:11 2022] PUT /identity/v3/projects/f3207a01779a4397b1beaebbc38530ba/users/d146463b8d944bb0ab517d0d613eef64/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:11.752364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924195a9-88f1-4382-b334-e3e36ea469b3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.752759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924195a9-88f1-4382-b334-e3e36ea469b3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.753180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924195a9-88f1-4382-b334-e3e36ea469b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.775488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:11.790857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-924195a9-88f1-4382-b334-e3e36ea469b3 None None] MFA Rules not processed for user `d146463b8d944bb0ab517d0d613eef64`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:11.792536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-924195a9-88f1-4382-b334-e3e36ea469b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:11.869154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1414/2812] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:11.881604 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab8057-ccc4-4669-ba83-a4fae45f6980 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:11.883761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3eab8057-ccc4-4669-ba83-a4fae45f6980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:11.934059 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:11.946850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab8057-ccc4-4669-ba83-a4fae45f6980 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:11.951193 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721b76b-88b5-4a1e-b5d6-acecf3c13032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:11.951842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721b76b-88b5-4a1e-b5d6-acecf3c13032 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:11.952256 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721b76b-88b5-4a1e-b5d6-acecf3c13032 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:11.952528 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721b76b-88b5-4a1e-b5d6-acecf3c13032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:11.963112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721b76b-88b5-4a1e-b5d6-acecf3c13032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:11.964538 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721b76b-88b5-4a1e-b5d6-acecf3c13032 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:11.974608 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1399/2813] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:12.055485 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924195a9-88f1-4382-b334-e3e36ea469b3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:12.065857 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924195a9-88f1-4382-b334-e3e36ea469b3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:12.070468 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4531e7d-f028-4299-ade0-304bfc8c7e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:12.071197 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4531e7d-f028-4299-ade0-304bfc8c7e50 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:12.071396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4531e7d-f028-4299-ade0-304bfc8c7e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:12.071693 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4531e7d-f028-4299-ade0-304bfc8c7e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:12.130703 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4531e7d-f028-4299-ade0-304bfc8c7e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:12.132148 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4531e7d-f028-4299-ade0-304bfc8c7e50 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:12.145268 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1415/2814] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:12.415192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721b76b-88b5-4a1e-b5d6-acecf3c13032 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:12.479706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721b76b-88b5-4a1e-b5d6-acecf3c13032 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:12.485148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:12.486026 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:12.486301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:12.486569 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] PATH_INFO: `/v3/users/6809f80b3362418696e88d3eafe95748` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:12.487800 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6809f80b3362418696e88d3eafe95748)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:12.489766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:12.545421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6809f80b3362418696e88d3eafe95748'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:12.557752 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6809f80b3362418696e88d3eafe95748'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:12.568573 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6809f80b3362418696e88d3eafe95748'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:12.576009 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6809f80b3362418696e88d3eafe95748'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:12.576665 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6809f80b3362418696e88d3eafe95748'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:12.582552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1400/2815] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:12 2022] DELETE /identity/v3/users/6809f80b3362418696e88d3eafe95748 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:12.587132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b43c23-e5f4-4ed1-8edb-219add1344b3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:12.587423 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b43c23-e5f4-4ed1-8edb-219add1344b3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:12.587706 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b43c23-e5f4-4ed1-8edb-219add1344b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:12.605332 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-43b43c23-e5f4-4ed1-8edb-219add1344b3 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:12.606518 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-43b43c23-e5f4-4ed1-8edb-219add1344b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:12.674500 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1416/2816] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:12.685695 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:12.687266 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:12.755103 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0d8d8-84d6-42ed-89c5-5b2abba83490 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:12.759731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d4fc9-fbbd-445d-be48-b6c2544c053b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:12.760761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d4fc9-fbbd-445d-be48-b6c2544c053b service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:12.761078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d4fc9-fbbd-445d-be48-b6c2544c053b service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:12.761356 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d4fc9-fbbd-445d-be48-b6c2544c053b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:12.771699 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d4fc9-fbbd-445d-be48-b6c2544c053b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:12.777064 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d4fc9-fbbd-445d-be48-b6c2544c053b service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:12.787551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1401/2817] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:13.087563 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b43c23-e5f4-4ed1-8edb-219add1344b3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:13.089312 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-43b43c23-e5f4-4ed1-8edb-219add1344b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:13.102882 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b43c23-e5f4-4ed1-8edb-219add1344b3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:13.108583 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:13.109558 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:13.109891 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:13.110172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] PATH_INFO: `/v3/projects/27432a933d2e45a3940fd491c50f66c3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:13.111444 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27432a933d2e45a3940fd491c50f66c3)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:13.113494 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:13.147926 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:13.163897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27432a933d2e45a3940fd491c50f66c3', 'request_id': 'req-15e9970c-107b-4943-9422-e361c98c1ca0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.174886 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27432a933d2e45a3940fd491c50f66c3', 'request_id': 'req-15e9970c-107b-4943-9422-e361c98c1ca0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.180759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27432a933d2e45a3940fd491c50f66c3', 'request_id': 'req-15e9970c-107b-4943-9422-e361c98c1ca0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.193024 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] The token cache is being invalidate because project 27432a933d2e45a3940fd491c50f66c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:13.193312 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.194022 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.195661 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1417/2818] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:13 2022] DELETE /identity/v3/projects/27432a933d2e45a3940fd491c50f66c3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:13.200083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d4fc9-fbbd-445d-be48-b6c2544c053b service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:13.209787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d4fc9-fbbd-445d-be48-b6c2544c053b service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:13.214455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:13.269113 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:13.269973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:13.270309 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:13.270558 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] PATH_INFO: `/v3/users/d146463b8d944bb0ab517d0d613eef64` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:13.271788 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d146463b8d944bb0ab517d0d613eef64)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:13.273740 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:13.325244 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd146463b8d944bb0ab517d0d613eef64'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.337560 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd146463b8d944bb0ab517d0d613eef64'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.346830 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd146463b8d944bb0ab517d0d613eef64'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.354523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd146463b8d944bb0ab517d0d613eef64'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.355205 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfc79d2a-4ac8-4ae3-8f43-be6a3a0280a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd146463b8d944bb0ab517d0d613eef64'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.361286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1402/2819] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:13 2022] DELETE /identity/v3/users/d146463b8d944bb0ab517d0d613eef64 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:13.642172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:13.643762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:13.657341 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9970c-107b-4943-9422-e361c98c1ca0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:13.716382 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:13.717283 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:13.717569 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:13.717844 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] PATH_INFO: `/v3/projects/f3207a01779a4397b1beaebbc38530ba` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:13.719061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3207a01779a4397b1beaebbc38530ba)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:13.721046 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:13.770091 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3207a01779a4397b1beaebbc38530ba', 'request_id': 'req-33ffc0a4-5b62-49db-894f-b300ad8cdb02'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.780533 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3207a01779a4397b1beaebbc38530ba', 'request_id': 'req-33ffc0a4-5b62-49db-894f-b300ad8cdb02'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.788275 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3207a01779a4397b1beaebbc38530ba', 'request_id': 'req-33ffc0a4-5b62-49db-894f-b300ad8cdb02'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.800294 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] The token cache is being invalidate because project f3207a01779a4397b1beaebbc38530ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:13.800577 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.801240 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-33ffc0a4-5b62-49db-894f-b300ad8cdb02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:13.802817 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1418/2820] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:13 2022] DELETE /identity/v3/projects/f3207a01779a4397b1beaebbc38530ba => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:13.808209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081150c6-dc9f-469f-899e-c19b3bf21225 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:13.808706 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081150c6-dc9f-469f-899e-c19b3bf21225 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:13.808839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081150c6-dc9f-469f-899e-c19b3bf21225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:13.829457 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-081150c6-dc9f-469f-899e-c19b3bf21225 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:13.830889 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-081150c6-dc9f-469f-899e-c19b3bf21225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:13.891217 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1403/2821] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:13.899128 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fdce9-d52d-4c66-ad82-471bf94476b7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:13.899415 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fdce9-d52d-4c66-ad82-471bf94476b7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:13.899688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fdce9-d52d-4c66-ad82-471bf94476b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:13.919209 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-793fdce9-d52d-4c66-ad82-471bf94476b7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:13.920400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-793fdce9-d52d-4c66-ad82-471bf94476b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:13.928615 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1419/2822] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:13.933005 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081150c6-dc9f-469f-899e-c19b3bf21225 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:13.942987 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081150c6-dc9f-469f-899e-c19b3bf21225 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:13.948146 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b1a46-5982-4db7-a197-deffe765a465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:13.948842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b1a46-5982-4db7-a197-deffe765a465 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:13.949095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b1a46-5982-4db7-a197-deffe765a465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:13.949383 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b1a46-5982-4db7-a197-deffe765a465 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:13.950108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b1a46-5982-4db7-a197-deffe765a465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:47:13.950410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b1a46-5982-4db7-a197-deffe765a465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:13.951995 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b1a46-5982-4db7-a197-deffe765a465 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:13.963990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1404/2823] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:47:13.968428 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fdce9-d52d-4c66-ad82-471bf94476b7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:13.978580 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fdce9-d52d-4c66-ad82-471bf94476b7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:13.984082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65836065-61d2-4284-9bd1-a6f8540d243c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:13.984749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65836065-61d2-4284-9bd1-a6f8540d243c admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:13.985042 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65836065-61d2-4284-9bd1-a6f8540d243c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:13.985317 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65836065-61d2-4284-9bd1-a6f8540d243c admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:13.986005 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65836065-61d2-4284-9bd1-a6f8540d243c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:13.987894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65836065-61d2-4284-9bd1-a6f8540d243c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:14.016325 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1420/2824] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:13 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:14.020222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b1a46-5982-4db7-a197-deffe765a465 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:14.030896 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b1a46-5982-4db7-a197-deffe765a465 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:14.042839 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:14.098761 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfacd4-d9f2-473f-98ec-23013cea6d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:14.099385 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfacd4-d9f2-473f-98ec-23013cea6d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:14.099668 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfacd4-d9f2-473f-98ec-23013cea6d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:14.099953 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfacd4-d9f2-473f-98ec-23013cea6d51 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:14.100661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfacd4-d9f2-473f-98ec-23013cea6d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:14.102653 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfacd4-d9f2-473f-98ec-23013cea6d51 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:14.127365 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1405/2825] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:14 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:14.132795 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65836065-61d2-4284-9bd1-a6f8540d243c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:14.150606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65836065-61d2-4284-9bd1-a6f8540d243c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:14.158170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dccb20-385b-4043-827d-f48b4bd5b25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:14.159255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dccb20-385b-4043-827d-f48b4bd5b25f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:14.159602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dccb20-385b-4043-827d-f48b4bd5b25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:14.159945 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dccb20-385b-4043-827d-f48b4bd5b25f admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:14.161260 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dccb20-385b-4043-827d-f48b4bd5b25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:14.163798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dccb20-385b-4043-827d-f48b4bd5b25f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:14.180667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1421/2826] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:14.186455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfacd4-d9f2-473f-98ec-23013cea6d51 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:14.197707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfacd4-d9f2-473f-98ec-23013cea6d51 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:14.203112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846ce13-c81a-49c2-8158-a30df4e70cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:14.203792 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846ce13-c81a-49c2-8158-a30df4e70cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:14.204076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846ce13-c81a-49c2-8158-a30df4e70cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:14.204350 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846ce13-c81a-49c2-8158-a30df4e70cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:14.205124 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846ce13-c81a-49c2-8158-a30df4e70cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:14.206728 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846ce13-c81a-49c2-8158-a30df4e70cbb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:14.217479 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1406/2827] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:14.221742 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dccb20-385b-4043-827d-f48b4bd5b25f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:14.232908 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dccb20-385b-4043-827d-f48b4bd5b25f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:14.239532 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904cc04-9f55-400a-8614-9836edc3a95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:14.240425 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904cc04-9f55-400a-8614-9836edc3a95c admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:14.240708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904cc04-9f55-400a-8614-9836edc3a95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:14.241028 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904cc04-9f55-400a-8614-9836edc3a95c admin admin] PATH_INFO: `/v3/projects/5aa53945f5884d4aaa87c69ad9b24bb9/users/18f0b19982744987b157a3a6b29b37c3/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:14.243450 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:14.259175 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904cc04-9f55-400a-8614-9836edc3a95c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aa53945f5884d4aaa87c69ad9b24bb9, user_id=18f0b19982744987b157a3a6b29b37c3, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:14.261932 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904cc04-9f55-400a-8614-9836edc3a95c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:14.277164 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1422/2828] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:14 2022] PUT /identity/v3/projects/5aa53945f5884d4aaa87c69ad9b24bb9/users/18f0b19982744987b157a3a6b29b37c3/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:14.288721 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14717a67-23cb-4171-8a93-e87332d8feb3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:14.289043 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14717a67-23cb-4171-8a93-e87332d8feb3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:14.289326 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14717a67-23cb-4171-8a93-e87332d8feb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:14.317192 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-14717a67-23cb-4171-8a93-e87332d8feb3 None None] MFA Rules not processed for user `18f0b19982744987b157a3a6b29b37c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:14.318416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-14717a67-23cb-4171-8a93-e87332d8feb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:14.386611 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1407/2829] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:14.397936 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904cc04-9f55-400a-8614-9836edc3a95c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:14.399710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e904cc04-9f55-400a-8614-9836edc3a95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:14.467244 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904cc04-9f55-400a-8614-9836edc3a95c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:14.471328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e1add-ad20-4ba0-a5c8-fe4dcd4db565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:14.472304 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e1add-ad20-4ba0-a5c8-fe4dcd4db565 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:14.472606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e1add-ad20-4ba0-a5c8-fe4dcd4db565 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:14.473009 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e1add-ad20-4ba0-a5c8-fe4dcd4db565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:14.482132 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e1add-ad20-4ba0-a5c8-fe4dcd4db565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:14.484404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e1add-ad20-4ba0-a5c8-fe4dcd4db565 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:14.494144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1423/2830] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:14.513512 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14717a67-23cb-4171-8a93-e87332d8feb3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:14.581678 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14717a67-23cb-4171-8a93-e87332d8feb3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:14.586967 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:14.587887 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:14.588200 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:14.588468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] PATH_INFO: `/v3/users/18f0b19982744987b157a3a6b29b37c3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:14.589926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f0b19982744987b157a3a6b29b37c3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:14.591873 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:14.623352 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:14.643976 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f0b19982744987b157a3a6b29b37c3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:14.655530 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f0b19982744987b157a3a6b29b37c3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:14.664434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f0b19982744987b157a3a6b29b37c3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:14.672012 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f0b19982744987b157a3a6b29b37c3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:14.672648 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f0b19982744987b157a3a6b29b37c3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:14.678567 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1408/2831] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:14 2022] DELETE /identity/v3/users/18f0b19982744987b157a3a6b29b37c3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:14.683391 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a2665-ab16-4921-a903-11f5f7666123 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:14.683701 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a2665-ab16-4921-a903-11f5f7666123 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:14.683982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a2665-ab16-4921-a903-11f5f7666123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:14.701935 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f12a2665-ab16-4921-a903-11f5f7666123 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:14.703150 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f12a2665-ab16-4921-a903-11f5f7666123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:14.774098 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1424/2832] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:47:14.785220 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:14.788787 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:14.850306 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2a303-5e5c-44e1-b424-5f437d287103 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:14.854667 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2d18b-c7ef-46b5-9f14-976a05f14485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:14.855311 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2d18b-c7ef-46b5-9f14-976a05f14485 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:14.855591 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2d18b-c7ef-46b5-9f14-976a05f14485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:14.855871 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2d18b-c7ef-46b5-9f14-976a05f14485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:14.865426 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2d18b-c7ef-46b5-9f14-976a05f14485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:14.866853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2d18b-c7ef-46b5-9f14-976a05f14485 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:14.876712 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1409/2833] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:15.168861 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a2665-ab16-4921-a903-11f5f7666123 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:15.170375 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f12a2665-ab16-4921-a903-11f5f7666123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:15.182650 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a2665-ab16-4921-a903-11f5f7666123 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:15.188053 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:15.188946 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.189297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.189645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] PATH_INFO: `/v3/projects/5aa53945f5884d4aaa87c69ad9b24bb9` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.190941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aa53945f5884d4aaa87c69ad9b24bb9)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:15.192979 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:15.244218 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aa53945f5884d4aaa87c69ad9b24bb9', 'request_id': 'req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:15.253642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aa53945f5884d4aaa87c69ad9b24bb9', 'request_id': 'req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:15.259356 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aa53945f5884d4aaa87c69ad9b24bb9', 'request_id': 'req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:15.272257 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] The token cache is being invalidate because project 5aa53945f5884d4aaa87c69ad9b24bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:15.272540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:15.273252 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a78f1cbf-d70d-4df7-a5d5-392514f2fcba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:15.274823 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1425/2834] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:15 2022] DELETE /identity/v3/projects/5aa53945f5884d4aaa87c69ad9b24bb9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:15.279888 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278ab15-000b-46ee-91cc-ff7214d34eec None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.280209 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278ab15-000b-46ee-91cc-ff7214d34eec None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.280513 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278ab15-000b-46ee-91cc-ff7214d34eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.299239 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e278ab15-000b-46ee-91cc-ff7214d34eec None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:15.300461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e278ab15-000b-46ee-91cc-ff7214d34eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:15.370903 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1410/2835] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:15.379070 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695a21b-40dd-458d-9722-e71784f0c7a7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.379364 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695a21b-40dd-458d-9722-e71784f0c7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.379645 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695a21b-40dd-458d-9722-e71784f0c7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.398542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b695a21b-40dd-458d-9722-e71784f0c7a7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:15.399768 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b695a21b-40dd-458d-9722-e71784f0c7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:15.408799 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1426/2836] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:15.413384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278ab15-000b-46ee-91cc-ff7214d34eec None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:15.426000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278ab15-000b-46ee-91cc-ff7214d34eec None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:15.431818 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6c1fb-9aae-48ee-ac0c-e28a5decd1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:15.432523 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6c1fb-9aae-48ee-ac0c-e28a5decd1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.432848 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6c1fb-9aae-48ee-ac0c-e28a5decd1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.433134 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6c1fb-9aae-48ee-ac0c-e28a5decd1e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.433874 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6c1fb-9aae-48ee-ac0c-e28a5decd1e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:47:15.434182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6c1fb-9aae-48ee-ac0c-e28a5decd1e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:15.435825 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6c1fb-9aae-48ee-ac0c-e28a5decd1e9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:15.448175 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1411/2837] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:47:15.452554 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695a21b-40dd-458d-9722-e71784f0c7a7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:15.457100 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:15.463047 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695a21b-40dd-458d-9722-e71784f0c7a7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:15.469642 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b7159-7525-409f-838f-b634946ddea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:15.470380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b7159-7525-409f-838f-b634946ddea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.470667 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b7159-7525-409f-838f-b634946ddea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.470948 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b7159-7525-409f-838f-b634946ddea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.471847 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b7159-7525-409f-838f-b634946ddea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:15.474547 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b7159-7525-409f-838f-b634946ddea7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:15.502671 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1427/2838] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:15 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:15.507020 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6c1fb-9aae-48ee-ac0c-e28a5decd1e9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:15.517387 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6c1fb-9aae-48ee-ac0c-e28a5decd1e9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:15.547407 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:15.574615 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fd888-f911-49c1-b891-4ec2cfa39156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:15.575325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fd888-f911-49c1-b891-4ec2cfa39156 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.575603 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fd888-f911-49c1-b891-4ec2cfa39156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.575875 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fd888-f911-49c1-b891-4ec2cfa39156 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.576587 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fd888-f911-49c1-b891-4ec2cfa39156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:15.578529 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fd888-f911-49c1-b891-4ec2cfa39156 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:15.600098 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1412/2839] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:15 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:15.606538 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b7159-7525-409f-838f-b634946ddea7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:15.617291 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b7159-7525-409f-838f-b634946ddea7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:15.622971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d8743-3944-4b58-9e46-9e66d27e7f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:15.623637 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d8743-3944-4b58-9e46-9e66d27e7f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.623923 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d8743-3944-4b58-9e46-9e66d27e7f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.624200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d8743-3944-4b58-9e46-9e66d27e7f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.624971 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d8743-3944-4b58-9e46-9e66d27e7f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:15.626576 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d8743-3944-4b58-9e46-9e66d27e7f1b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:15.636826 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1428/2840] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:15.640868 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fd888-f911-49c1-b891-4ec2cfa39156 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:15.650980 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fd888-f911-49c1-b891-4ec2cfa39156 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:15.655966 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512650c-8237-4fa3-807e-d8d30cdd8d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:15.656631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512650c-8237-4fa3-807e-d8d30cdd8d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.656946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512650c-8237-4fa3-807e-d8d30cdd8d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.657224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512650c-8237-4fa3-807e-d8d30cdd8d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.657954 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512650c-8237-4fa3-807e-d8d30cdd8d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:15.659540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512650c-8237-4fa3-807e-d8d30cdd8d60 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:15.670449 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1413/2841] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:15.674458 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d8743-3944-4b58-9e46-9e66d27e7f1b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:15.684470 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d8743-3944-4b58-9e46-9e66d27e7f1b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:15.691431 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3caa3-8457-46a1-9629-e27b8d17c8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:15.692855 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3caa3-8457-46a1-9629-e27b8d17c8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.693300 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3caa3-8457-46a1-9629-e27b8d17c8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.693756 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3caa3-8457-46a1-9629-e27b8d17c8ab admin admin] PATH_INFO: `/v3/projects/def1c4c11f894df2a3174f8f79c96746/users/ca9fe04cfc45465fb3dcc97689d7c5c1/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.717254 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3caa3-8457-46a1-9629-e27b8d17c8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=def1c4c11f894df2a3174f8f79c96746, user_id=ca9fe04cfc45465fb3dcc97689d7c5c1, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:15.719876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3caa3-8457-46a1-9629-e27b8d17c8ab admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:15.736141 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1429/2842] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:15 2022] PUT /identity/v3/projects/def1c4c11f894df2a3174f8f79c96746/users/ca9fe04cfc45465fb3dcc97689d7c5c1/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:15.747133 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b2a98-f86a-44e6-80aa-6ee8becf3924 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.747446 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b2a98-f86a-44e6-80aa-6ee8becf3924 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.747724 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b2a98-f86a-44e6-80aa-6ee8becf3924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.774904 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ed3b2a98-f86a-44e6-80aa-6ee8becf3924 None None] MFA Rules not processed for user `ca9fe04cfc45465fb3dcc97689d7c5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:15.776143 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ed3b2a98-f86a-44e6-80aa-6ee8becf3924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:15.855332 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1414/2843] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:15.861457 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3caa3-8457-46a1-9629-e27b8d17c8ab admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:15.929515 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3caa3-8457-46a1-9629-e27b8d17c8ab admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:15.934746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccd19f-4934-4a8b-9d02-f613ca1e59e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:15.935397 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccd19f-4934-4a8b-9d02-f613ca1e59e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:15.935675 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccd19f-4934-4a8b-9d02-f613ca1e59e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:15.935944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccd19f-4934-4a8b-9d02-f613ca1e59e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:15.936626 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccd19f-4934-4a8b-9d02-f613ca1e59e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:15.938586 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccd19f-4934-4a8b-9d02-f613ca1e59e9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:15.966363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1430/2844] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:15 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:47:15.970532 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b2a98-f86a-44e6-80aa-6ee8becf3924 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:15.982148 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b2a98-f86a-44e6-80aa-6ee8becf3924 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.042095 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd2a3f-9a1a-44ca-a54c-40013b023d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.042911 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd2a3f-9a1a-44ca-a54c-40013b023d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.043191 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd2a3f-9a1a-44ca-a54c-40013b023d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.043465 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd2a3f-9a1a-44ca-a54c-40013b023d45 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.044169 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd2a3f-9a1a-44ca-a54c-40013b023d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.046088 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd2a3f-9a1a-44ca-a54c-40013b023d45 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.067224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1415/2845] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:15 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:47:16.072652 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccd19f-4934-4a8b-9d02-f613ca1e59e9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.083095 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccd19f-4934-4a8b-9d02-f613ca1e59e9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.087951 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2f663-5d45-47b6-9709-38923d40e488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.088610 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2f663-5d45-47b6-9709-38923d40e488 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.088950 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2f663-5d45-47b6-9709-38923d40e488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.089231 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2f663-5d45-47b6-9709-38923d40e488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.089970 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2f663-5d45-47b6-9709-38923d40e488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.091565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2f663-5d45-47b6-9709-38923d40e488 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.102347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1431/2846] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:16.106545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd2a3f-9a1a-44ca-a54c-40013b023d45 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.116872 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd2a3f-9a1a-44ca-a54c-40013b023d45 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.121890 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859928-512f-4eec-8da6-a3214c99d2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.122563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859928-512f-4eec-8da6-a3214c99d2ba admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.122838 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859928-512f-4eec-8da6-a3214c99d2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.123110 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859928-512f-4eec-8da6-a3214c99d2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.123867 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54859928-512f-4eec-8da6-a3214c99d2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.125461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54859928-512f-4eec-8da6-a3214c99d2ba admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.135657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1416/2847] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:16.139759 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2f663-5d45-47b6-9709-38923d40e488 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.149740 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2f663-5d45-47b6-9709-38923d40e488 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.154872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c9922-00f7-43b4-8cdd-3a9cd71195c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.155746 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c9922-00f7-43b4-8cdd-3a9cd71195c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.156021 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c9922-00f7-43b4-8cdd-3a9cd71195c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.156297 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c9922-00f7-43b4-8cdd-3a9cd71195c3 admin admin] PATH_INFO: `/v3/projects/777359abc94a49cab8ba592302529f98/users/e18bfa7e6fd4411e85a3851a4d5c2b6f/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.171002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c9922-00f7-43b4-8cdd-3a9cd71195c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=777359abc94a49cab8ba592302529f98, user_id=e18bfa7e6fd4411e85a3851a4d5c2b6f, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.173668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c9922-00f7-43b4-8cdd-3a9cd71195c3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.188525 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1432/2848] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:16 2022] PUT /identity/v3/projects/777359abc94a49cab8ba592302529f98/users/e18bfa7e6fd4411e85a3851a4d5c2b6f/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:47:16.253153 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bc115-b3cc-41d1-bf32-df95746f659e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.253388 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bc115-b3cc-41d1-bf32-df95746f659e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.253586 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bc115-b3cc-41d1-bf32-df95746f659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.281145 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b24bc115-b3cc-41d1-bf32-df95746f659e None None] MFA Rules not processed for user `e18bfa7e6fd4411e85a3851a4d5c2b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:16.282345 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b24bc115-b3cc-41d1-bf32-df95746f659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:16.291142 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:16.350343 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1417/2849] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:16.355418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c9922-00f7-43b4-8cdd-3a9cd71195c3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.415565 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:16.420991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c9922-00f7-43b4-8cdd-3a9cd71195c3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.426542 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23d9f6-98dd-4e59-bad3-962da7fa34e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.427196 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23d9f6-98dd-4e59-bad3-962da7fa34e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.427476 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23d9f6-98dd-4e59-bad3-962da7fa34e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.427767 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23d9f6-98dd-4e59-bad3-962da7fa34e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.428424 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23d9f6-98dd-4e59-bad3-962da7fa34e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.430359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23d9f6-98dd-4e59-bad3-962da7fa34e8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.457094 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1433/2850] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:16 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:16.461289 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bc115-b3cc-41d1-bf32-df95746f659e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.472760 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bc115-b3cc-41d1-bf32-df95746f659e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.541841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0864c-16ee-4e06-b7f5-721b3906efb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.542631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0864c-16ee-4e06-b7f5-721b3906efb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.542910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0864c-16ee-4e06-b7f5-721b3906efb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.543181 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0864c-16ee-4e06-b7f5-721b3906efb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.543866 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0864c-16ee-4e06-b7f5-721b3906efb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.545797 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0864c-16ee-4e06-b7f5-721b3906efb3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.566934 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1418/2851] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:16 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:16.570997 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23d9f6-98dd-4e59-bad3-962da7fa34e8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.581166 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23d9f6-98dd-4e59-bad3-962da7fa34e8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.585803 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e1758-c2c5-4474-9bad-9e95980212cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.586472 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e1758-c2c5-4474-9bad-9e95980212cb admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.586752 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e1758-c2c5-4474-9bad-9e95980212cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.586998 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e1758-c2c5-4474-9bad-9e95980212cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.587711 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e1758-c2c5-4474-9bad-9e95980212cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.589744 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e1758-c2c5-4474-9bad-9e95980212cb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.600972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1434/2852] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:16.605128 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0864c-16ee-4e06-b7f5-721b3906efb3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.615384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0864c-16ee-4e06-b7f5-721b3906efb3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.620470 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7435fd-00e5-4600-98c2-180c7f74b4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.621373 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7435fd-00e5-4600-98c2-180c7f74b4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.621659 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7435fd-00e5-4600-98c2-180c7f74b4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.621932 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7435fd-00e5-4600-98c2-180c7f74b4c5 admin admin] PATH_INFO: `/v3/projects/41128160f32d41859f0fea4742d4fd63/users/6d2029e4111c4b699b5cb95efc7b36bd/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.637156 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7435fd-00e5-4600-98c2-180c7f74b4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41128160f32d41859f0fea4742d4fd63, user_id=6d2029e4111c4b699b5cb95efc7b36bd, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.639847 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7435fd-00e5-4600-98c2-180c7f74b4c5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.655631 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1419/2853] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:16 2022] PUT /identity/v3/projects/41128160f32d41859f0fea4742d4fd63/users/6d2029e4111c4b699b5cb95efc7b36bd/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:16.659765 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e1758-c2c5-4474-9bad-9e95980212cb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.670228 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e1758-c2c5-4474-9bad-9e95980212cb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.728646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04388ee3-0fe6-4fff-9b0f-511d082cf1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.729407 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04388ee3-0fe6-4fff-9b0f-511d082cf1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.729708 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04388ee3-0fe6-4fff-9b0f-511d082cf1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.729980 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04388ee3-0fe6-4fff-9b0f-511d082cf1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.730710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04388ee3-0fe6-4fff-9b0f-511d082cf1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.732312 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04388ee3-0fe6-4fff-9b0f-511d082cf1d2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.742748 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1435/2854] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:47:16.747662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7435fd-00e5-4600-98c2-180c7f74b4c5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.758713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7435fd-00e5-4600-98c2-180c7f74b4c5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.764506 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32899086-cf47-4200-ab64-88668a04608a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.765463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32899086-cf47-4200-ab64-88668a04608a admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.765746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32899086-cf47-4200-ab64-88668a04608a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.766024 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32899086-cf47-4200-ab64-88668a04608a admin admin] PATH_INFO: `/v3/projects/41128160f32d41859f0fea4742d4fd63/users/6d2029e4111c4b699b5cb95efc7b36bd/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.768318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32899086-cf47-4200-ab64-88668a04608a admin admin] RBAC: Authorizing `identity:create_grant(project_id=41128160f32d41859f0fea4742d4fd63, user_id=6d2029e4111c4b699b5cb95efc7b36bd, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:16.771001 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32899086-cf47-4200-ab64-88668a04608a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:16.782822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1420/2855] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:16 2022] PUT /identity/v3/projects/41128160f32d41859f0fea4742d4fd63/users/6d2029e4111c4b699b5cb95efc7b36bd/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:16.793647 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470f284-7dff-4933-9f61-edb0e8d82f3c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.793865 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470f284-7dff-4933-9f61-edb0e8d82f3c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.794123 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470f284-7dff-4933-9f61-edb0e8d82f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:16.828359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f470f284-7dff-4933-9f61-edb0e8d82f3c None None] MFA Rules not processed for user `6d2029e4111c4b699b5cb95efc7b36bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:16.829611 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f470f284-7dff-4933-9f61-edb0e8d82f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:16.911710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1436/2856] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:47:16.923633 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32899086-cf47-4200-ab64-88668a04608a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:16.925436 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-32899086-cf47-4200-ab64-88668a04608a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:16.984096 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32899086-cf47-4200-ab64-88668a04608a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:16.987807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180ced4-bb04-47e6-a0d0-542de0018811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:16.988442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180ced4-bb04-47e6-a0d0-542de0018811 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:16.988717 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180ced4-bb04-47e6-a0d0-542de0018811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:16.989028 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180ced4-bb04-47e6-a0d0-542de0018811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:17.001574 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:17.048666 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180ced4-bb04-47e6-a0d0-542de0018811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:17.050063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180ced4-bb04-47e6-a0d0-542de0018811 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:17.059807 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1421/2857] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:18.552312 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470f284-7dff-4933-9f61-edb0e8d82f3c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:18.553875 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f470f284-7dff-4933-9f61-edb0e8d82f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:18.566894 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470f284-7dff-4933-9f61-edb0e8d82f3c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:18.570770 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104390f-68ab-42dd-936d-5216e46b30fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:18.571412 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104390f-68ab-42dd-936d-5216e46b30fd service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:18.571687 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104390f-68ab-42dd-936d-5216e46b30fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:18.572077 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104390f-68ab-42dd-936d-5216e46b30fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:18.620385 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:18.630814 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104390f-68ab-42dd-936d-5216e46b30fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:18.632281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104390f-68ab-42dd-936d-5216e46b30fd service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:18.642627 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1437/2858] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:19.201451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180ced4-bb04-47e6-a0d0-542de0018811 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:19.215750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180ced4-bb04-47e6-a0d0-542de0018811 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:19.222376 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fcc28-6ff9-4fc0-8b2e-7972baf9103a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5586ef8925ef4cc496d358283c21c962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:19.223302 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fcc28-6ff9-4fc0-8b2e-7972baf9103a service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:19.223594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fcc28-6ff9-4fc0-8b2e-7972baf9103a service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:19.223898 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fcc28-6ff9-4fc0-8b2e-7972baf9103a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:19.236538 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fcc28-6ff9-4fc0-8b2e-7972baf9103a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:19.238021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fcc28-6ff9-4fc0-8b2e-7972baf9103a service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:19.248279 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1422/2859] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:23.146163 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104390f-68ab-42dd-936d-5216e46b30fd service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:23.224011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104390f-68ab-42dd-936d-5216e46b30fd service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:23.229910 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:23.230778 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:23.231071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:23.231350 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] PATH_INFO: `/v3/users/ca9fe04cfc45465fb3dcc97689d7c5c1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:23.232665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9fe04cfc45465fb3dcc97689d7c5c1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:23.234640 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:23.287710 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9fe04cfc45465fb3dcc97689d7c5c1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.300466 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9fe04cfc45465fb3dcc97689d7c5c1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.310067 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9fe04cfc45465fb3dcc97689d7c5c1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.310668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9fe04cfc45465fb3dcc97689d7c5c1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.317618 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9fe04cfc45465fb3dcc97689d7c5c1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.323570 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1438/2860] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:23 2022] DELETE /identity/v3/users/ca9fe04cfc45465fb3dcc97689d7c5c1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:23.328292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61be985-8603-4246-b6d5-e3bbcf0fb4b7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:23.328842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61be985-8603-4246-b6d5-e3bbcf0fb4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:23.328842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61be985-8603-4246-b6d5-e3bbcf0fb4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:23.347410 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c61be985-8603-4246-b6d5-e3bbcf0fb4b7 None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:23.348628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c61be985-8603-4246-b6d5-e3bbcf0fb4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:23.409305 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1423/2861] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:47:23.419421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:23.421140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:23.480118 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027bed8-2629-4b81-a42c-8a52c1eafd29 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:23.484126 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305c5f1-15db-4e38-ae45-c89e6ee3b049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:23.484774 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305c5f1-15db-4e38-ae45-c89e6ee3b049 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:23.485071 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305c5f1-15db-4e38-ae45-c89e6ee3b049 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:23.485344 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305c5f1-15db-4e38-ae45-c89e6ee3b049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:23.494343 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305c5f1-15db-4e38-ae45-c89e6ee3b049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:23.495832 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305c5f1-15db-4e38-ae45-c89e6ee3b049 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:23.505156 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1439/2862] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:23.802076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61be985-8603-4246-b6d5-e3bbcf0fb4b7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:23.803682 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c61be985-8603-4246-b6d5-e3bbcf0fb4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:23.816105 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61be985-8603-4246-b6d5-e3bbcf0fb4b7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:23.821597 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93175b33-8844-491f-a435-c555149b254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:23.822468 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93175b33-8844-491f-a435-c555149b254a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:23.822780 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93175b33-8844-491f-a435-c555149b254a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:23.823021 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93175b33-8844-491f-a435-c555149b254a admin admin] PATH_INFO: `/v3/projects/def1c4c11f894df2a3174f8f79c96746` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:23.824524 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93175b33-8844-491f-a435-c555149b254a admin admin] RBAC: Authorizing `identity:delete_project(project_id=def1c4c11f894df2a3174f8f79c96746)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:23.826543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93175b33-8844-491f-a435-c555149b254a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:23.878641 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-93175b33-8844-491f-a435-c555149b254a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def1c4c11f894df2a3174f8f79c96746', 'request_id': 'req-93175b33-8844-491f-a435-c555149b254a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.885053 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-93175b33-8844-491f-a435-c555149b254a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def1c4c11f894df2a3174f8f79c96746', 'request_id': 'req-93175b33-8844-491f-a435-c555149b254a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.894627 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-93175b33-8844-491f-a435-c555149b254a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def1c4c11f894df2a3174f8f79c96746', 'request_id': 'req-93175b33-8844-491f-a435-c555149b254a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.905709 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-93175b33-8844-491f-a435-c555149b254a admin admin] The token cache is being invalidate because project def1c4c11f894df2a3174f8f79c96746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:23.905998 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-93175b33-8844-491f-a435-c555149b254a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.906652 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-93175b33-8844-491f-a435-c555149b254a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:23.908232 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1424/2863] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:23 2022] DELETE /identity/v3/projects/def1c4c11f894df2a3174f8f79c96746 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:23.912416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305c5f1-15db-4e38-ae45-c89e6ee3b049 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:23.922481 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305c5f1-15db-4e38-ae45-c89e6ee3b049 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:23.932337 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:23.985941 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:23.986825 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:23.987107 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:23.987380 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] PATH_INFO: `/v3/users/e18bfa7e6fd4411e85a3851a4d5c2b6f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:23.988595 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18bfa7e6fd4411e85a3851a4d5c2b6f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:23.990555 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:24.041648 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18bfa7e6fd4411e85a3851a4d5c2b6f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.053177 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18bfa7e6fd4411e85a3851a4d5c2b6f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.061681 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18bfa7e6fd4411e85a3851a4d5c2b6f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.062416 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18bfa7e6fd4411e85a3851a4d5c2b6f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.069900 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18bfa7e6fd4411e85a3851a4d5c2b6f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.076496 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1440/2864] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:23 2022] DELETE /identity/v3/users/e18bfa7e6fd4411e85a3851a4d5c2b6f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:47:24.368195 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93175b33-8844-491f-a435-c555149b254a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:24.369707 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-93175b33-8844-491f-a435-c555149b254a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:24.383727 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93175b33-8844-491f-a435-c555149b254a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:24.412576 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:24.447657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:24.448501 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:24.448809 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:24.449086 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] PATH_INFO: `/v3/projects/777359abc94a49cab8ba592302529f98` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:24.450292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] RBAC: Authorizing `identity:delete_project(project_id=777359abc94a49cab8ba592302529f98)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:24.452221 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:24.503127 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777359abc94a49cab8ba592302529f98', 'request_id': 'req-4e1ac190-6b92-48c3-b220-ea06b5eb82da'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.509182 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777359abc94a49cab8ba592302529f98', 'request_id': 'req-4e1ac190-6b92-48c3-b220-ea06b5eb82da'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.518813 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777359abc94a49cab8ba592302529f98', 'request_id': 'req-4e1ac190-6b92-48c3-b220-ea06b5eb82da'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.529214 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] The token cache is being invalidate because project 777359abc94a49cab8ba592302529f98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:24.529497 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.530120 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.531669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1425/2865] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:24 2022] DELETE /identity/v3/projects/777359abc94a49cab8ba592302529f98 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:24.535720 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:24.537200 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:24.549840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aed5f0-9230-45d1-8f11-62647e3bb16e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:24.608641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:24.609546 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:24.609830 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:24.610109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] PATH_INFO: `/v3/users/6d2029e4111c4b699b5cb95efc7b36bd` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:24.611328 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2029e4111c4b699b5cb95efc7b36bd)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:24.613295 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:24.653991 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:24.668725 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2029e4111c4b699b5cb95efc7b36bd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.680606 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2029e4111c4b699b5cb95efc7b36bd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.689961 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2029e4111c4b699b5cb95efc7b36bd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.690620 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2029e4111c4b699b5cb95efc7b36bd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.697758 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4cb7092a-1df4-46f1-acd3-4756c7b71669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2029e4111c4b699b5cb95efc7b36bd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:24.703715 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1441/2866] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:47:24 2022] DELETE /identity/v3/users/6d2029e4111c4b699b5cb95efc7b36bd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:47:24.992947 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:24.994429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:25.007981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ac190-6b92-48c3-b220-ea06b5eb82da admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:25.067669 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:25.068540 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.068850 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.069113 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] PATH_INFO: `/v3/projects/41128160f32d41859f0fea4742d4fd63` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.070271 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41128160f32d41859f0fea4742d4fd63)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:25.072216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:25.121908 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41128160f32d41859f0fea4742d4fd63', 'request_id': 'req-4e9da4f5-1003-4cca-a944-80f840ae9703'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:25.127861 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41128160f32d41859f0fea4742d4fd63', 'request_id': 'req-4e9da4f5-1003-4cca-a944-80f840ae9703'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:25.131427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:25.139297 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41128160f32d41859f0fea4742d4fd63', 'request_id': 'req-4e9da4f5-1003-4cca-a944-80f840ae9703'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:25.151458 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] The token cache is being invalidate because project 41128160f32d41859f0fea4742d4fd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:47:25.151748 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:25.152447 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e9da4f5-1003-4cca-a944-80f840ae9703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:47:25.154031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1426/2867] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:47:24 2022] DELETE /identity/v3/projects/41128160f32d41859f0fea4742d4fd63 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:25.159129 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5567-a5de-4be7-98f0-0b6d0213cd9b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.159421 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5567-a5de-4be7-98f0-0b6d0213cd9b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.159699 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5567-a5de-4be7-98f0-0b6d0213cd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.178749 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-293c5567-a5de-4be7-98f0-0b6d0213cd9b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:25.180076 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-293c5567-a5de-4be7-98f0-0b6d0213cd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:25.243438 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1442/2868] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:47:25.253844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad96e6b-9375-48db-9660-4f418a8c5a5b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.254393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad96e6b-9375-48db-9660-4f418a8c5a5b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.254814 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad96e6b-9375-48db-9660-4f418a8c5a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.275585 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bad96e6b-9375-48db-9660-4f418a8c5a5b None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:25.276929 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bad96e6b-9375-48db-9660-4f418a8c5a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:25.285926 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1427/2869] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:25.290312 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c5567-a5de-4be7-98f0-0b6d0213cd9b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:25.300398 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c5567-a5de-4be7-98f0-0b6d0213cd9b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:25.305316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daf26f-5635-4077-90d4-f56d9149ddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:25.305972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf26f-5635-4077-90d4-f56d9149ddc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.306255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf26f-5635-4077-90d4-f56d9149ddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.306536 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf26f-5635-4077-90d4-f56d9149ddc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.307267 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07daf26f-5635-4077-90d4-f56d9149ddc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:47:25.307660 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07daf26f-5635-4077-90d4-f56d9149ddc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:25.309255 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07daf26f-5635-4077-90d4-f56d9149ddc6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:25.320599 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1443/2870] 10.222.0.69 () {58 vars in 1152 bytes} [Fri Jun 24 10:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:47:25.324990 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad96e6b-9375-48db-9660-4f418a8c5a5b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:25.334384 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad96e6b-9375-48db-9660-4f418a8c5a5b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:25.340246 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472fe57-d2bc-4ce9-8c51-5fd1c20dab19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:25.340946 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472fe57-d2bc-4ce9-8c51-5fd1c20dab19 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.341230 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472fe57-d2bc-4ce9-8c51-5fd1c20dab19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.341581 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472fe57-d2bc-4ce9-8c51-5fd1c20dab19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.342334 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472fe57-d2bc-4ce9-8c51-5fd1c20dab19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:25.344612 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472fe57-d2bc-4ce9-8c51-5fd1c20dab19 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:25.375413 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1428/2871] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:25 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:25.378540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daf26f-5635-4077-90d4-f56d9149ddc6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:25.394292 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daf26f-5635-4077-90d4-f56d9149ddc6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:25.456688 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf96a6-0a0f-42b2-95dc-f597a9bf46c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:25.457400 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf96a6-0a0f-42b2-95dc-f597a9bf46c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.457683 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf96a6-0a0f-42b2-95dc-f597a9bf46c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.458084 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf96a6-0a0f-42b2-95dc-f597a9bf46c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.458665 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf96a6-0a0f-42b2-95dc-f597a9bf46c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:25.460602 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf96a6-0a0f-42b2-95dc-f597a9bf46c1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:25.484109 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1444/2872] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:25 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:47:25.488480 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472fe57-d2bc-4ce9-8c51-5fd1c20dab19 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:25.499565 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472fe57-d2bc-4ce9-8c51-5fd1c20dab19 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:25.504927 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a156a53-14ca-42d0-ac3d-c8a4ef1d5b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:25.505575 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a156a53-14ca-42d0-ac3d-c8a4ef1d5b49 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.505855 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a156a53-14ca-42d0-ac3d-c8a4ef1d5b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.506126 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a156a53-14ca-42d0-ac3d-c8a4ef1d5b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.506853 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a156a53-14ca-42d0-ac3d-c8a4ef1d5b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:25.508451 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a156a53-14ca-42d0-ac3d-c8a4ef1d5b49 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:25.519548 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1429/2873] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:25.523527 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf96a6-0a0f-42b2-95dc-f597a9bf46c1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:25.533876 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf96a6-0a0f-42b2-95dc-f597a9bf46c1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:25.538995 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2a76d-57c7-4e4a-ad99-c7955172377f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:25.539612 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2a76d-57c7-4e4a-ad99-c7955172377f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.539914 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2a76d-57c7-4e4a-ad99-c7955172377f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.540185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2a76d-57c7-4e4a-ad99-c7955172377f admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.540949 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2a76d-57c7-4e4a-ad99-c7955172377f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:25.542540 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2a76d-57c7-4e4a-ad99-c7955172377f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:25.552888 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1445/2874] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:25.557172 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a156a53-14ca-42d0-ac3d-c8a4ef1d5b49 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:25.567596 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a156a53-14ca-42d0-ac3d-c8a4ef1d5b49 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:25.572763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762de7d6-71ff-4eb5-a94e-fbd33e65efcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:25.573718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762de7d6-71ff-4eb5-a94e-fbd33e65efcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.574000 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762de7d6-71ff-4eb5-a94e-fbd33e65efcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.574298 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762de7d6-71ff-4eb5-a94e-fbd33e65efcc admin admin] PATH_INFO: `/v3/projects/9d15300203df4c139c86b1ddf43f34c4/users/75b2d6e8e9614ce49ed02d63493b6612/roles/9ae7e8c79b724c70890cf352b9748307` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.589010 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762de7d6-71ff-4eb5-a94e-fbd33e65efcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d15300203df4c139c86b1ddf43f34c4, user_id=75b2d6e8e9614ce49ed02d63493b6612, role_id=9ae7e8c79b724c70890cf352b9748307)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:25.591661 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762de7d6-71ff-4eb5-a94e-fbd33e65efcc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:25.607535 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1430/2875] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:25 2022] PUT /identity/v3/projects/9d15300203df4c139c86b1ddf43f34c4/users/75b2d6e8e9614ce49ed02d63493b6612/roles/9ae7e8c79b724c70890cf352b9748307 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:25.613043 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d60f22-0127-4104-a84d-6cce797638fb None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.613331 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d60f22-0127-4104-a84d-6cce797638fb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.613641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d60f22-0127-4104-a84d-6cce797638fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.632639 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a8d60f22-0127-4104-a84d-6cce797638fb None None] MFA Rules not processed for user `86fea4d6526441fa9d310035bfcc8012`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:25.633872 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a8d60f22-0127-4104-a84d-6cce797638fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:25.643769 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:25.696404 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1446/2876] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:25.706429 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762de7d6-71ff-4eb5-a94e-fbd33e65efcc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:25.708203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-762de7d6-71ff-4eb5-a94e-fbd33e65efcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:25.769906 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762de7d6-71ff-4eb5-a94e-fbd33e65efcc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:25.774509 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebaf0d-26fd-4781-a246-64a5ca76f095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:25.775161 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebaf0d-26fd-4781-a246-64a5ca76f095 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:25.775442 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebaf0d-26fd-4781-a246-64a5ca76f095 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:25.775713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebaf0d-26fd-4781-a246-64a5ca76f095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:25.786314 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebaf0d-26fd-4781-a246-64a5ca76f095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:25.787744 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebaf0d-26fd-4781-a246-64a5ca76f095 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:25.797733 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1431/2877] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:47:29.738296 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f4f1d-5f9d-447e-996d-7c30d118ae7e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:29.738885 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f4f1d-5f9d-447e-996d-7c30d118ae7e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:29.739002 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f4f1d-5f9d-447e-996d-7c30d118ae7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:29.778213 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bf5f4f1d-5f9d-447e-996d-7c30d118ae7e None None] MFA Rules not processed for user `75b2d6e8e9614ce49ed02d63493b6612`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:29.779172 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bf5f4f1d-5f9d-447e-996d-7c30d118ae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:29.849194 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1447/2878] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:29.853427 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebaf0d-26fd-4781-a246-64a5ca76f095 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:29.864058 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebaf0d-26fd-4781-a246-64a5ca76f095 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:29.868758 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a54b5-5cce-4675-bd9f-1003f53e58db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:29.869461 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a54b5-5cce-4675-bd9f-1003f53e58db admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:29.869745 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a54b5-5cce-4675-bd9f-1003f53e58db admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:29.870260 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a54b5-5cce-4675-bd9f-1003f53e58db admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:29.870975 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a54b5-5cce-4675-bd9f-1003f53e58db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:29.872973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a54b5-5cce-4675-bd9f-1003f53e58db admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:29.904910 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1432/2879] 10.222.0.69 () {60 vars in 1152 bytes} [Fri Jun 24 10:47:29 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:47:29.906045 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f4f1d-5f9d-447e-996d-7c30d118ae7e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:29.917363 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f4f1d-5f9d-447e-996d-7c30d118ae7e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:29.974968 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ae4e4-bbf8-4648-9b4f-a3742e4ba69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:29.975668 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ae4e4-bbf8-4648-9b4f-a3742e4ba69b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:29.975989 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ae4e4-bbf8-4648-9b4f-a3742e4ba69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:29.976271 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ae4e4-bbf8-4648-9b4f-a3742e4ba69b admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:29.977012 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ae4e4-bbf8-4648-9b4f-a3742e4ba69b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:29.978992 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ae4e4-bbf8-4648-9b4f-a3742e4ba69b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:30.001798 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1448/2880] 10.222.0.69 () {60 vars in 1143 bytes} [Fri Jun 24 10:47:29 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:47:30.007225 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a54b5-5cce-4675-bd9f-1003f53e58db admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:30.018734 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a54b5-5cce-4675-bd9f-1003f53e58db admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:30.023406 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b563e99-f50d-4804-9555-9e16eda89abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:30.024083 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b563e99-f50d-4804-9555-9e16eda89abb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:30.024369 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b563e99-f50d-4804-9555-9e16eda89abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:30.024649 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b563e99-f50d-4804-9555-9e16eda89abb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:30.025420 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b563e99-f50d-4804-9555-9e16eda89abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:30.027218 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b563e99-f50d-4804-9555-9e16eda89abb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:30.038731 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1433/2881] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:30.043613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ae4e4-bbf8-4648-9b4f-a3742e4ba69b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:30.054562 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ae4e4-bbf8-4648-9b4f-a3742e4ba69b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:30.059456 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c8a5-182d-46d9-bcf5-620365bbbc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:30.060361 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c8a5-182d-46d9-bcf5-620365bbbc64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:30.060641 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c8a5-182d-46d9-bcf5-620365bbbc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:30.060944 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c8a5-182d-46d9-bcf5-620365bbbc64 admin admin] PATH_INFO: `/v3/projects/c9b72c205b204bdeacdfa364ac4fc2be/users/0ac3d11cf00a4eb08e8abd755219cd73/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:30.077808 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49c8a5-182d-46d9-bcf5-620365bbbc64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b72c205b204bdeacdfa364ac4fc2be, user_id=0ac3d11cf00a4eb08e8abd755219cd73, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:30.080491 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49c8a5-182d-46d9-bcf5-620365bbbc64 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:30.095519 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1449/2882] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:30 2022] PUT /identity/v3/projects/c9b72c205b204bdeacdfa364ac4fc2be/users/0ac3d11cf00a4eb08e8abd755219cd73/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:30.100112 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b563e99-f50d-4804-9555-9e16eda89abb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:30.110166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b563e99-f50d-4804-9555-9e16eda89abb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:30.157594 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:30.168551 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e4609-5eda-49dc-831e-c2813726442e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:30.169422 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e4609-5eda-49dc-831e-c2813726442e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:30.169923 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e4609-5eda-49dc-831e-c2813726442e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:30.170203 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e4609-5eda-49dc-831e-c2813726442e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:30.170945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e4609-5eda-49dc-831e-c2813726442e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:30.172543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e4609-5eda-49dc-831e-c2813726442e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:30.182735 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1434/2883] 10.222.0.69 () {58 vars in 1121 bytes} [Fri Jun 24 10:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:47:30.187088 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c8a5-182d-46d9-bcf5-620365bbbc64 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:30.197656 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c8a5-182d-46d9-bcf5-620365bbbc64 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:30.203590 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b9da8-65f4-454c-b52f-007761341302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:30.204473 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b9da8-65f4-454c-b52f-007761341302 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:30.204723 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b9da8-65f4-454c-b52f-007761341302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:30.205061 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b9da8-65f4-454c-b52f-007761341302 admin admin] PATH_INFO: `/v3/projects/c9b72c205b204bdeacdfa364ac4fc2be/users/0ac3d11cf00a4eb08e8abd755219cd73/roles/fff8f769b9bf46018c80bd5d274e7888` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:30.207281 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b9da8-65f4-454c-b52f-007761341302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b72c205b204bdeacdfa364ac4fc2be, user_id=0ac3d11cf00a4eb08e8abd755219cd73, role_id=fff8f769b9bf46018c80bd5d274e7888)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:30.209984 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b9da8-65f4-454c-b52f-007761341302 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:30.216140 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:30.225017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1450/2884] 10.222.0.69 () {60 vars in 1482 bytes} [Fri Jun 24 10:47:30 2022] PUT /identity/v3/projects/c9b72c205b204bdeacdfa364ac4fc2be/users/0ac3d11cf00a4eb08e8abd755219cd73/roles/fff8f769b9bf46018c80bd5d274e7888 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:47:34.616199 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaa55e-9643-4b3a-a57a-f3f0c10a91bc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:34.616766 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaa55e-9643-4b3a-a57a-f3f0c10a91bc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:34.617194 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaa55e-9643-4b3a-a57a-f3f0c10a91bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:34.664027 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-eebaa55e-9643-4b3a-a57a-f3f0c10a91bc None None] MFA Rules not processed for user `0ac3d11cf00a4eb08e8abd755219cd73`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:34.666031 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eebaa55e-9643-4b3a-a57a-f3f0c10a91bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:34.804992 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1435/2885] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:34.817198 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b9da8-65f4-454c-b52f-007761341302 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:34.820104 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-526b9da8-65f4-454c-b52f-007761341302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:34.918374 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b9da8-65f4-454c-b52f-007761341302 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:34.924205 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0f902-0891-4196-a147-c89e710ea6d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:34.924903 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0f902-0891-4196-a147-c89e710ea6d4 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:34.925193 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0f902-0891-4196-a147-c89e710ea6d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:34.925475 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0f902-0891-4196-a147-c89e710ea6d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:34.997056 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0f902-0891-4196-a147-c89e710ea6d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:34.998982 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0f902-0891-4196-a147-c89e710ea6d4 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:35.010011 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1451/2886] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:47:36.042842 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fc15f-4af6-482d-82d1-bd54e363cc6a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:36.043341 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fc15f-4af6-482d-82d1-bd54e363cc6a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:36.043443 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fc15f-4af6-482d-82d1-bd54e363cc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:36.061941 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cd7fc15f-4af6-482d-82d1-bd54e363cc6a None None] MFA Rules not processed for user `75b2d6e8e9614ce49ed02d63493b6612`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:47:36.063173 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cd7fc15f-4af6-482d-82d1-bd54e363cc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:36.070725 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1436/2887] 10.222.0.69 () {58 vars in 957 bytes} [Fri Jun 24 10:47:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:47:45.005152 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=57 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:46.056325 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=42 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:47:50.897336 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0f902-0891-4196-a147-c89e710ea6d4 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:50.953878 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0f902-0891-4196-a147-c89e710ea6d4 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:50.957892 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:50.958534 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:50.958815 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:50.959092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:50.961117 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:50.971092 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:50.973001 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:50.981287 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1452/2888] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:47:51.443418 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fc15f-4af6-482d-82d1-bd54e363cc6a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:51.445076 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cd7fc15f-4af6-482d-82d1-bd54e363cc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:51.518762 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fc15f-4af6-482d-82d1-bd54e363cc6a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:51.524168 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273abf5-a153-4f9a-8ffa-04b64f26408c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:51.524841 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273abf5-a153-4f9a-8ffa-04b64f26408c service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:51.525147 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273abf5-a153-4f9a-8ffa-04b64f26408c service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:51.525416 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273abf5-a153-4f9a-8ffa-04b64f26408c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:51.527264 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0273abf5-a153-4f9a-8ffa-04b64f26408c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:51.538822 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273abf5-a153-4f9a-8ffa-04b64f26408c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:51.540392 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273abf5-a153-4f9a-8ffa-04b64f26408c service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:51.550393 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1437/2889] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:47:53.955972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:47:53.957657 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:53.982967 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131206d-99ae-4769-b0de-e00aae6119e2 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:47:53.989601 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54ffe4-ee63-4650-a41d-13732ebaa688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:47:53.990310 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54ffe4-ee63-4650-a41d-13732ebaa688 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:47:53.990607 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54ffe4-ee63-4650-a41d-13732ebaa688 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:47:53.990863 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54ffe4-ee63-4650-a41d-13732ebaa688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:47:53.992784 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fd54ffe4-ee63-4650-a41d-13732ebaa688 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:47:54.008673 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54ffe4-ee63-4650-a41d-13732ebaa688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:47:54.011153 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54ffe4-ee63-4650-a41d-13732ebaa688 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:47:54.023188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1453/2890] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:47:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:48:01.546286 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=32 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:48:04.018636 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=32 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:48:04.706981 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273abf5-a153-4f9a-8ffa-04b64f26408c service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:48:04.753328 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273abf5-a153-4f9a-8ffa-04b64f26408c service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:48:04.759336 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c3fdb-1a0e-48a0-9159-08cf62286ce9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:48:04.760022 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c3fdb-1a0e-48a0-9159-08cf62286ce9 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:48:04.760301 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c3fdb-1a0e-48a0-9159-08cf62286ce9 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:48:04.760572 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c3fdb-1a0e-48a0-9159-08cf62286ce9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:48:04.762363 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-724c3fdb-1a0e-48a0-9159-08cf62286ce9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:48:04.793662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c3fdb-1a0e-48a0-9159-08cf62286ce9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:48:04.795166 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c3fdb-1a0e-48a0-9159-08cf62286ce9 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:48:04.805723 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1438/2891] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:48:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:48:07.163977 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54ffe4-ee63-4650-a41d-13732ebaa688 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:48:07.185488 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54ffe4-ee63-4650-a41d-13732ebaa688 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:48:07.189474 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a690d-8fbd-40f1-9db5-d2da40901dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:48:07.190111 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a690d-8fbd-40f1-9db5-d2da40901dfe service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:48:07.190369 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a690d-8fbd-40f1-9db5-d2da40901dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:48:07.190662 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a690d-8fbd-40f1-9db5-d2da40901dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:48:07.200020 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a690d-8fbd-40f1-9db5-d2da40901dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:48:07.201492 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a690d-8fbd-40f1-9db5-d2da40901dfe service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:48:07.211170 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1454/2892] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:48:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:48:14.801718 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=28 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:48:17.207154 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=11 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:48:39.579935 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c3fdb-1a0e-48a0-9159-08cf62286ce9 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:48:39.612079 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c3fdb-1a0e-48a0-9159-08cf62286ce9 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:48:39.618344 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec12e47-ed69-4b29-bd59-d0632c3e36ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:48:39.619003 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec12e47-ed69-4b29-bd59-d0632c3e36ef service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:48:39.619285 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec12e47-ed69-4b29-bd59-d0632c3e36ef service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:48:39.619563 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec12e47-ed69-4b29-bd59-d0632c3e36ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:48:39.621464 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5ec12e47-ed69-4b29-bd59-d0632c3e36ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:48:39.649227 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec12e47-ed69-4b29-bd59-d0632c3e36ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:48:39.650844 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec12e47-ed69-4b29-bd59-d0632c3e36ef service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:48:39.661622 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1439/2893] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:48:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:48:49.657070 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=15 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:50:17.467858 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a690d-8fbd-40f1-9db5-d2da40901dfe service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:50:17.487664 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a690d-8fbd-40f1-9db5-d2da40901dfe service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:50:17.491824 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3a7dc-555f-4e5e-aed4-6103cdb65c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:50:17.492471 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3a7dc-555f-4e5e-aed4-6103cdb65c1a service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:50:17.492843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3a7dc-555f-4e5e-aed4-6103cdb65c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:50:17.493120 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3a7dc-555f-4e5e-aed4-6103cdb65c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:50:17.495144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-11d3a7dc-555f-4e5e-aed4-6103cdb65c1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:50:17.518313 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3a7dc-555f-4e5e-aed4-6103cdb65c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:50:17.520169 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3a7dc-555f-4e5e-aed4-6103cdb65c1a service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:50:17.529017 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1455/2894] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:50:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:50:17.635552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec12e47-ed69-4b29-bd59-d0632c3e36ef service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:50:17.645577 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec12e47-ed69-4b29-bd59-d0632c3e36ef service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:50:17.649774 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797aac1a-533e-4138-882d-c05e0d5f17c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:50:17.650425 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aac1a-533e-4138-882d-c05e0d5f17c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:50:17.650713 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aac1a-533e-4138-882d-c05e0d5f17c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:50:17.650991 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aac1a-533e-4138-882d-c05e0d5f17c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:50:17.652899 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-797aac1a-533e-4138-882d-c05e0d5f17c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:50:17.686750 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797aac1a-533e-4138-882d-c05e0d5f17c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:50:17.688108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797aac1a-533e-4138-882d-c05e0d5f17c3 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:50:17.709318 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1440/2895] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:50:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:50:27.525015 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=15 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:50:27.706434 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=18 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:51:37.983316 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3a7dc-555f-4e5e-aed4-6103cdb65c1a service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:37.996270 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3a7dc-555f-4e5e-aed4-6103cdb65c1a service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:38.001048 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605f145-fbd9-491d-8384-b3d7592c761a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:38.002185 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605f145-fbd9-491d-8384-b3d7592c761a service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:38.002785 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605f145-fbd9-491d-8384-b3d7592c761a service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:38.003262 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605f145-fbd9-491d-8384-b3d7592c761a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:38.007029 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a605f145-fbd9-491d-8384-b3d7592c761a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:51:38.025305 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605f145-fbd9-491d-8384-b3d7592c761a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:38.027027 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605f145-fbd9-491d-8384-b3d7592c761a service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:38.039619 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1456/2896] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:51:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:51:38.917919 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797aac1a-533e-4138-882d-c05e0d5f17c3 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:38.928255 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797aac1a-533e-4138-882d-c05e0d5f17c3 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:38.932545 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378ec01-1663-4310-9873-580610f841b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:38.933223 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378ec01-1663-4310-9873-580610f841b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:38.933505 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378ec01-1663-4310-9873-580610f841b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:38.933778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378ec01-1663-4310-9873-580610f841b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:38.935957 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a378ec01-1663-4310-9873-580610f841b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:51:38.947140 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378ec01-1663-4310-9873-580610f841b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:38.948562 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378ec01-1663-4310-9873-580610f841b3 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:38.956859 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1441/2897] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:51:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:51:39.246144 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605f145-fbd9-491d-8384-b3d7592c761a service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:39.258229 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605f145-fbd9-491d-8384-b3d7592c761a service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:39.262843 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d1b25-a3df-418c-9d11-4554498d6288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86de22b0e324c70aad792ec6b1a7e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:39.263482 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d1b25-a3df-418c-9d11-4554498d6288 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:39.263762 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d1b25-a3df-418c-9d11-4554498d6288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:39.264031 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d1b25-a3df-418c-9d11-4554498d6288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:39.266359 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-789d1b25-a3df-418c-9d11-4554498d6288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:51:39.281389 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d1b25-a3df-418c-9d11-4554498d6288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:39.282816 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d1b25-a3df-418c-9d11-4554498d6288 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:39.293884 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1457/2898] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:51:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:51:39.859747 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378ec01-1663-4310-9873-580610f841b3 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:39.874628 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378ec01-1663-4310-9873-580610f841b3 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:39.881108 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b656e-ee5d-4db0-8b4c-5a5a60c14e33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec9fee282a64933aa0f6fbdd91a93c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:39.881775 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b656e-ee5d-4db0-8b4c-5a5a60c14e33 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:39.882063 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b656e-ee5d-4db0-8b4c-5a5a60c14e33 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:39.882330 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b656e-ee5d-4db0-8b4c-5a5a60c14e33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:39.884292 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-577b656e-ee5d-4db0-8b4c-5a5a60c14e33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:51:39.898602 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b656e-ee5d-4db0-8b4c-5a5a60c14e33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:39.900137 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b656e-ee5d-4db0-8b4c-5a5a60c14e33 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:39.911595 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1442/2899] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:51:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:51:45.983224 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d1b25-a3df-418c-9d11-4554498d6288 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:46.012498 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d1b25-a3df-418c-9d11-4554498d6288 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:46.019897 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e96ab-00d6-4442-83b4-7e4051e05f60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:46.020579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e96ab-00d6-4442-83b4-7e4051e05f60 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:46.020916 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e96ab-00d6-4442-83b4-7e4051e05f60 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:46.021384 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e96ab-00d6-4442-83b4-7e4051e05f60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:46.025290 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6d4e96ab-00d6-4442-83b4-7e4051e05f60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:51:46.038134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e96ab-00d6-4442-83b4-7e4051e05f60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:46.039735 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e96ab-00d6-4442-83b4-7e4051e05f60 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:46.059004 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1458/2900] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:51:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:51:49.201224 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b656e-ee5d-4db0-8b4c-5a5a60c14e33 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:49.213693 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b656e-ee5d-4db0-8b4c-5a5a60c14e33 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:49.219455 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6dba1-cfd4-439f-b27b-a4a3e5b41809 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:49.220250 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6dba1-cfd4-439f-b27b-a4a3e5b41809 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:49.220676 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6dba1-cfd4-439f-b27b-a4a3e5b41809 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:49.221007 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6dba1-cfd4-439f-b27b-a4a3e5b41809 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:49.222845 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0bc6dba1-cfd4-439f-b27b-a4a3e5b41809 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:51:49.239421 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6dba1-cfd4-439f-b27b-a4a3e5b41809 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:49.241552 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6dba1-cfd4-439f-b27b-a4a3e5b41809 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:49.258134 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e96ab-00d6-4442-83b4-7e4051e05f60 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:49.263493 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1443/2901] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:51:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:51:49.351579 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e96ab-00d6-4442-83b4-7e4051e05f60 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:49.356463 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:49.357368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:49.357646 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:49.357918 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] PATH_INFO: `/v3/users/75b2d6e8e9614ce49ed02d63493b6612` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:49.359719 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75b2d6e8e9614ce49ed02d63493b6612)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:49.362840 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:49.413868 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75b2d6e8e9614ce49ed02d63493b6612'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.426741 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75b2d6e8e9614ce49ed02d63493b6612'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.435649 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75b2d6e8e9614ce49ed02d63493b6612'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.436418 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75b2d6e8e9614ce49ed02d63493b6612'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.444347 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75b2d6e8e9614ce49ed02d63493b6612'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.457862 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1459/2902] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:51:49 2022] DELETE /identity/v3/users/75b2d6e8e9614ce49ed02d63493b6612 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:51:49.638973 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6dba1-cfd4-439f-b27b-a4a3e5b41809 service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:49.651213 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6dba1-cfd4-439f-b27b-a4a3e5b41809 service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:49.708786 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:49.709685 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:49.709965 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:49.710241 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] PATH_INFO: `/v3/projects/9d15300203df4c139c86b1ddf43f34c4` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:49.711463 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d15300203df4c139c86b1ddf43f34c4)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:49.713502 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:49.763754 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d15300203df4c139c86b1ddf43f34c4', 'request_id': 'req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.770135 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d15300203df4c139c86b1ddf43f34c4', 'request_id': 'req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.780078 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d15300203df4c139c86b1ddf43f34c4', 'request_id': 'req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.803507 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] The token cache is being invalidate because project 9d15300203df4c139c86b1ddf43f34c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:51:49.803785 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.804484 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.806138 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1444/2903] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:51:49 2022] DELETE /identity/v3/projects/9d15300203df4c139c86b1ddf43f34c4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:51:49.810368 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:49.811850 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:51:49.825396 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fa7e3-e9a9-4854-9629-240ad0b89d90 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:49.845180 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:51:49.884613 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:49.885550 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:49.885831 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:49.886103 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] PATH_INFO: `/v3/users/0ac3d11cf00a4eb08e8abd755219cd73` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:49.887320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac3d11cf00a4eb08e8abd755219cd73)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:49.889320 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:49.939520 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac3d11cf00a4eb08e8abd755219cd73'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.951837 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac3d11cf00a4eb08e8abd755219cd73'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.960789 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac3d11cf00a4eb08e8abd755219cd73'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.961479 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac3d11cf00a4eb08e8abd755219cd73'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.969340 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac3d11cf00a4eb08e8abd755219cd73'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:49.976055 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1460/2904] 10.222.0.69 () {58 vars in 1223 bytes} [Fri Jun 24 10:51:49 2022] DELETE /identity/v3/users/0ac3d11cf00a4eb08e8abd755219cd73 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:51:50.153662 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:51:50.155216 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:51:50.171878 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65be1fe-e22c-4a2e-9f7b-19c3c893c935 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:51:50.212566 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:51:50.232778 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86fea4d6526441fa9d310035bfcc8012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f9dd8794ad7494bbf369d7c8ce42c78', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:51:50.233657 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:51:50.233945 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:51:50.234222 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] PATH_INFO: `/v3/projects/c9b72c205b204bdeacdfa364ac4fc2be` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:51:50.235431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b72c205b204bdeacdfa364ac4fc2be)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:51:50.237380 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:51:50.294711 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b72c205b204bdeacdfa364ac4fc2be', 'request_id': 'req-0e186ad9-08d3-49f1-9c67-fd3022877f3a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:50.300922 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b72c205b204bdeacdfa364ac4fc2be', 'request_id': 'req-0e186ad9-08d3-49f1-9c67-fd3022877f3a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:50.310444 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b72c205b204bdeacdfa364ac4fc2be', 'request_id': 'req-0e186ad9-08d3-49f1-9c67-fd3022877f3a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:50.321763 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] The token cache is being invalidate because project c9b72c205b204bdeacdfa364ac4fc2be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:51:50.322051 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:50.322746 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0e186ad9-08d3-49f1-9c67-fd3022877f3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:51:50.324543 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1445/2905] 10.222.0.69 () {58 vars in 1232 bytes} [Fri Jun 24 10:51:50 2022] DELETE /identity/v3/projects/c9b72c205b204bdeacdfa364ac4fc2be => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:51:59.972052 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:52:00.318431 dv-cinder-847536-1-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:53:49.547745 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:53:49.549480 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:53:49.625972 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74a5a-1691-432f-bed7-ccd8f1b6c8f7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:53:49.689137 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ef7d4-039b-4350-8eee-d3b6228a3f04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373333a1fd64852a138b3f5913168c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6fbbd17a9ff41e98d76358258a07b80', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:53:49.689800 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ef7d4-039b-4350-8eee-d3b6228a3f04 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:53:49.690082 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ef7d4-039b-4350-8eee-d3b6228a3f04 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:53:49.690357 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ef7d4-039b-4350-8eee-d3b6228a3f04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:53:49.692339 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f73ef7d4-039b-4350-8eee-d3b6228a3f04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:53:49.703937 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ef7d4-039b-4350-8eee-d3b6228a3f04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:53:49.705545 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ef7d4-039b-4350-8eee-d3b6228a3f04 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:53:49.716188 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1461/2906] 10.222.0.69 () {60 vars in 1365 bytes} [Fri Jun 24 10:53:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:53:59.711743 dv-cinder-847536-1-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=58 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}